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CHAPTER  ONE: 
INTRODUCTION 

Since  the  beginning  of  time,  man  has  yearned  for  the 
ability  to  predict  the  future.  If  the  future  could  be 
predicted  accurately,  appropriate  plans  could  be  made  to 
prepare  for  future  circumstances.  From  a  modern  business 
standpoint,  the  ability  to  predict  the  future  would  mean 
that  the  organization  would  be  ready  to  utilize  state-of- 
the-art  technology  as  soon  as  it  is  available  and  could  make 
the  necessary  plans  to  meet  the  future  demands  of  its 
customers.  Without  such  abilities,  though,  organizations 
can  only  make  educated  guesses  in  their  attempt  to  plan  for 
the  future . 

It  is  this  preparation  for  the  future  that  is  the  very 
essence  of  research  and  development.  Research  and  develop- 
ment (R&D) ,  by  definition,  deals  with  new  ideas  and  the 
potential  technologies  that  will  form  the  basis  for  future 
products  and  processes.  Therefore,  the  success  of  an 
organization's  research  and  development  program  will 
determine  the  ability  of  the  organization  to  meet  the  needs 
of  its  customers  in  the  future.  Thus,  every  organization 
must  attempt  to  predict  the  future  to  insure  the  continua- 
tion of  its  operations.  To  complicate  this  problem,  the  act 
of  predicting  and  planning  for   the   future   effects   the 


future.  In  other  words,  the  rate  of  development  of  new 
technologies  is  directly  related  to  the  level  of  resources 
allocated  to  this  development. 

The  cumulative  effect  of  these  organizational  impacts 
can  be  seen  in  the  performance  of  an  entire  industry-  For 
example,  the  failure  of  individual  U.S.  steel  firms  to 
develop  more  efficient  processing  technologies  has  caused 
the  industry  as  a  whole  to  suffer  great  losses  at  the 
hands  of  more  efficient  suppliers.  These  industry  effects, 
in  turn,  have  an  impact  on  the  economy  as  a  whole. 

Consequently,  the  total  amount  of  research  and  develop- 
ment performed  in  this  country  is  commonly  considered  to  be 
an  indicator  of  the  long-run  health  of  the  economy.  In  an 
economy  tied  to  short-run  measures  such  as  quarterly  profit 
statements,  research  and  development  provides  a  measure  of 
the  performance  of  the  economy  5  to  10  years  from  now. 

A  survey  conducted  on  behalf  of  the  National  Science 
Foundation  by  the  Industrial  Research  Institute  concluded 
that  there  has  been  a  marked  decrease  in  the  amount  of  basic 
research  performed  in  this  country  in  the  last  two  decades 
[N-2].  Although  some  link  this  fact  to  the  weakened  compet- 
itiveness of  U.S.  industry,  it  is  more  likely  that  the  lack 
of  emphasis  on  applied  research  or  development  is  a  more 
direct  cause  of  this  decline.  It  is  generally  accepted  that 
the  U.S.  has  the  technology  to  improve  its  production 
efficiency  but  that  other  countries'  implementation  of   the 


technology  has  been  quicker  and  more  widespread.  Clearly,  a 
delicate  balance  of  basic  research  and  development  must  be 
maintained  for  a  healthy  economy. 

As  can  be  seen  from  the  previous  discussion,  the 
appropriate  allocation  of  resources  to  research  and 
development  is  important  to  individual  organizations  and  to 
the  economy  as  a  whole.  This  importance  is  reflected  in  the 
amount  of  literature  focused  on  the  research  and  development 
project  selection  problem.  The  process  of  R&D  project 
selection  is  the  realization  of  the  total  R&D  process.  It 
involves  the  determination  of  the  best  means  (R&D  projects) 
by  which  the  organization  prepares  for  the  future  and  the 
actual  allocation  of  resources  to  these  projects. 

This  report  will  focus  on  this  key  step  in  the  research 
and  development  process.  First,  Chapter  Two  will  define 
research  and  development  and  what  is  meant  by  an  R&D  project 
selection  decision.  The  definitions  of  both  R&D  and  the  R&D 
project  selection  process  will  be  seen  to  vary  according  to 
the  setting  in  which  the  projects  are  being  considered. 

Chapter  Three  will  begin  with  a  discussion  of  the 
actual  use  of  management  science  models  for  R&D  project 
selection.  Although  many  models  exist,  it  will  be  seen  that 
most  organizations  use  only  basic  economic  models  or  no 
organized  selection  process  at  all.  This  chapter  also 
traces  the  evolution  of  modeling  from  simple  checklists 
through  mathematical  programming  models  and  project   eval- 


uation  techniques  to  a  more  recent  emphasis  on  group 
decision  making.  An  example  of  one  of  these  "modern" 
project  selection  models  will  be  presented. 

After  demonstrating  the  importance  of  R&D  to  individual 
organizations  and  to  the  economy  as  a  whole  in  Chapter  One, 
describing  the  many  complexities  involved  in  the  R&D  project 
selection  process  in  Chapter  Two,  and,  in  Chapter  Three, 
realizing  the  inadequacy  of  the  models  proposed  in 
literature,  the  need  for  further  work  in  this  area  becomes 
evident.  An  R&D  manager  who  turns  to  the  literature  in 
hopes  of  improving  his/her  own  selection  system  will  find 
only  a  myriad  of  individual  approaches  to  the  problem  and 
little  guidance  on  how  choose  among  them.  The  next  three 
chapters,  then,  are  designed  to  organize  this  information 
into  a  simple,  logical  form  that  can  be  immediately 
understood  and  applied  by  an  R&D  manager  trying  to  improve 
his/her  own  R&D  project  selection  process. 

Chapter  Four  proposes  a  unique  approach  to  the  problem 
which  combines  group  decision  making,  the  classification  of 
R&D  projects,  and  the  use  of  current  project  selection 
models.  This  four-phase  model  begins  with  the  formulation 
of  statements  of  the  goals  of  the  organization. 
Classification  of  R&D  projects  is  proposed  to  facilitate  the 
use  of  many  current  selection  models  which  are  only 
appropriate  for  certain  types  of  projects.  The  final  phase 
consists   of  using  a  portfolio  model  as  an  analytical  guide 


to  a  committee  of  decision  makers  responsible  for  the  final 
allocation  of  resources  among  projects. 

Chapter  Five  discusses  a  number  of  classification 
schemes  for  R&D  projects  to  be  used  in  the  above  system. 
The  schemes  presented  in  this  chapter  are  those  that  are 
most  quoted  in  the  literature  and  most  commonly  used  in 
practice.  It  must  be  emphasized,  though,  that  the  actual 
classification  scheme  used  by  an  organization  must  be  based 
on  the  specific  goals  of  that  organization. 

Chapter  Six  presents  a  discussion  of  the  organizational 
factors  involved  in  implementing  the  proposed  system.  The 
system  is  designed  to  be  a  guide  for  the  development  of  a 
selection  system  for  all  organizations  and  can  be  adapted 
for  different  organizational  types,  structures,  and  sizes. 

Finally,  Chapter  Seven  summarizes  the  advantages 
associated  with  the  use  of  the  proposed  selection  system. 
The  classification  of  projects  allows  for  the  more 
appropriate  use  of  project  selection  models  and  for  insuring 
a  more  "balanced"  R&D  portfolio.  The  combination  of  a  group 
decision  making  technique  and  an  analytical  model  offers  the 
advantages  of  both  of  these  tools.  The  analytical  model 
helps  decision  makers  better  analyze  trade-offs  between 
projects  and  the  group  decision  making  technique  offers 
additional  advantages  such  as  improved  communication,  coop- 
eration, and  motivation.  The  system  recognizes  that  each 
organization  has  different  goals  and   alternative  projects 


and   is  designed  to  adapt  to  these  differences   yet  provide 
guidance  in  the  selection  process. 


CHAPTER  TWO: 
THE  R&D  PROJECT  SELECTION  PROCESS 

Any  single  definition  of  the  research  and  development 
project  selection  process  is  certain  to  be  subject  to 
scrutiny  from  academicians  and  practitioners  alike.  The 
lack  of  any  universal  understanding  of  what  is  meant  by 
"research  and  development,"  ignoring  any  understanding  of 
the  selection  process,  is  one  of  the  characteristics  that 
makes  the  discussion  of  R&D  project  selection  inherently 
complex. 

The  phrase  "research  and  development"  has  a  different 
meaning  for  every  individual  who  defines  it.  Some  people 
just  have  a  "feeling"  for  the  phrase  that  they  have  never 
put  into  words.  Some  define  the  phrase  as  one  entity  while 
others  define  "research"  separately  from  "development." 
Further,  any  attempt  to  draw  a  line  between  the  two  terms  is 
highly  subjective  since  the  actual  R&D  process  is  continuous 
rather  than  discrete. 

Further,  the  process  of  R&D  project  selection  is  a 
phrase  which  takes  on  widely  diverse  meanings  depending  on 
who  defines  it.  Each  individual's  understanding  of  the 
process  is  based  on  the  particular  environment  to  which 
he/she  has  been  exposed.  It  could  be  said  that  the  range  of 
definitions  derives   from  the   great   range   of   selection 


practices.  But,  it  could  just  as  well  be  that  the  range  of 
practices  is  the  result  of  the  lack  of  a  single  definition. 
Either  way,  definitions  range  from  the  application  of  a 
simple  economic  evaluation  method  to  complex  models  of  the 
total  innovative  process. 

To  simplify  the  discussion  in  this  report,  the 
definition  of  the  phrase  "research  and  development"  will  be 
discussed  followed  by  a  detailed  description  and  discussion 
of  what  is  meant  by  the  "R&D  project  selection  process." 

Definition  of  Research  and  Development  (R&D) 

The  phrase  "research  and  development"  is  commonly  used 
by  not  only  technical  personnel  but  by  the  public  at  large. 
As  with  most  commonly  used  phrases,  the  meaning  often 
changes  each  time  the  phrase  is  repeated.  Almost  everyone 
has  been,  at  one  time,  exposed  to  the  phrase.  Many 
Americans  are  concerned  with  the  guality  of  American  R&D 
verses  Japanese  R&D.  Investors  often  consider  the  amount 
spent  on  R&D  by  a  firm  as  an  indicator  of  the  future 
earnings  of  a  firm.  Further,  the  importance  of  R&D  to  a 
firm  is  evidenced  by  the  involvement  of  almost  every 
department  of  the  typical  business  unit:  accounting, 
finance,  marketing,  sales,  production,  as  well  as  the  actual 
R&D  unit  itself. 

With  each  of  these  groups   and   individuals  defining 


research  and  development  to  match  their  own  understanding 
and  point  of  view,  the  confusion  underlying  any  attempt  to 
come  up  with  a  single  definition  is  understandable.  But 
since  a  definition  is  necessary  for  a  full  discussion  of 
this  topic,  a  definition  will  be  presented  that  is  detailed 
enough  to  provide  the  basis  for  a  common  understanding  of 
the  phrase  and  general  enough  to  encompass  most  common 
points  of  view. 

A  very  general  definition  of  the  overall  objective  of 
R&D  is  presented  by  Balderston,  et  al.  [B-6]  as  follows: 


Research,  development,  and  engineering  involves 
the  continuous  expansion  of  organized  human 
knowledge,  and  the  application  of  that  human 
knowledge  in  devising  new  approaches  to  satisfying 
human  needs. 


Thus,  research  and  development  is  the  act  of  gathering  and 
applying  new  knowledge.  A  more  detailed  definition  involves 
differentiating  between  "research"  and  "development."  This 
division  is  the  subject  of  great  controversy  for  which  there 
is  no  resolve.  One  such  division  is  presented  by  Souder  [S- 
7]  who  differentiates  between  research  and  development  as 
follows: 


Research  is  that  set  of  activities  which 
identifies  the  new  product  or  process  and 
establishes   its  parameters   and  characteristics. 

Development  is  that  set  of  activities  which  rounds 
out  the  new  product  or  process  and  brings  it  to 
commercial  readiness. 


Understandably,  the  distinction  between  research  and 
development  in  this  and  most  definitions  is  weakly  defined. 
No  consensus  will  ever  be  reached  as  to  the  "best"  way  of 
dividing  this  continuous  process  into  a  finite  number  of 
stages.  The  issue  will  be  dealt  with  in  much  more  detail  in 
Chapter  Four,  Section  A,  which  discusses  the  classification 
of  R&D  projects  by  project  life  cycle. 

Accountants  have  a  unique  necessity  for  a  more 
practical  definition  of  research  and  development.  In  order 
to  properly  classify  expenses  as  research  and  development 
versus  other  business  expenditures,  simple  guidelines  must 
be  devised.  For  this  purpose,  the  Financial  Accounting 
Standards  Board  has  established  the  guidelines  for 
identifying  R&D  activities  as  presented  in  Table  1.  The 
first  list  includes  activities  that  are  typically  included 
in  the  category  of  R&D.  The  second  list  includes  those 
activities  that  are  typically  excluded  from  the  category  of 
R&D. 

These  lists  of  activities  help  to  develop  a  more 
practical  sense  of  the  meaning  of  research  and  development. 
Only  the  activities  associated  with  the  discovery  and 
testing  of  new  technologies  or  the  testing  of  new 
applications  of  existing  knowledge  are  included  within  the 
category  of  R&D.  Any  use  of  these  new  technologies  in 
activities  involved  in  regular  commercial  production  is 
considered   to  be   outside   the   realm  of   research   and 
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TABLE  1:   Financial  Accounting  Standards  Board's  R&D 
Guidelines.  [1-1] 

Activities  Typically  Included  in  R&D 

1.  Laboratory  research  aimed  at  discovery  of  new  knowledge. 

2.  Searching  for  applications  of  new  research  findings  or 
other  knowledge . 

3.  Conceptual  formulation  of  design  of  possible  product  or 
process  alternatives. 

4.  Testing  in  search  for  or  evaluation  of  product  or  process 
alternatives. 

5.  Modification  of  the  formulation  or  design  of  a  product  or 
process . 

6.  Design,  construction,  and  testing  of  pre-production 
prototypes  and  models. 

7.  Design  of  tools,  jigs,  molds,  and  dies  involving  new 
technology. 

8.  Design,  construction,  and  operation  of  a  pilot  plant  that 
is  not  of  a  scale  economically  feasible  to  the  enterprise 
for  commercial  production. 

9.  Engineering  activity  required  to  advance  the  design  of  a 
product  to  the  point  that  it  meets  specific  functional 
and  economic  requirements  and  is  ready  for  manufacture. 

Activities  Typically  Excluded  from  R&D 

1.  Engineering  follow-through  in  an  early  phase  of 
commercial  production. 

2.  Quality  control  during  commercial  production  including 
routine  testing  of  products. 

3 .  Trouble-shooting  in  connection  with  break-downs  during 
commercial  production. 

4.  Routine,  on-going  efforts  to  refine,  enrich,  or  otherwise 
improve  upon  the  qualities  of  an  existing  product. 

5.  Adaptation  of  an  existing  capability  to  a  particular 
requirement  or  customer's  need  as  part  of  a  continuing 
commercial  activity. 

6.  Seasonal  or  other  periodic  design  changes  to  existing 
products. 

7.  Routine  design  of  tools,  jigs,  molds,  and  dies. 

8.  Activity,  including  design  and  construction  engineering, 
related  to  the  construction,  relocation,  rearrangement, 
or  start-up  of  facilities  or  equipment.  Pilot  plants  of 
a  scale  economically  feasible  to  the  enterprise  for 
commercial  production. 

9.  Legal  work  in  connection  with  patent  applications  or 
litigation,  and  the  sale  or  licensing  of  patents. 
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development. 

In  summary,  endless  definitions  exist  for  the  phrase 
"research  and  development,"  depending  upon  the  individual 
and  the  environment  in  which  the  phrase  is  defined.  For 
discussion,  the  phrase  will  be  loosely  defined  as  "the 
discovery  and  application  of  new  knowledge  and  the  new 
application  of  existing  knowledge."  Based  on  this  under- 
standing, the  subject  of  the  R&D  project  selection  process 
can  now  be  addressed. 

The  Selection  Process 

The  quantity  of  literature  on  the  subject  of  research 
and  development  project  selection  reflects  not  only  the 
importance  of  the  topic  but  also  the  inherent  complexity  of 
this  unique  decision  process.  Although  many  models  and 
decision  systems  have  been  proposed  to  deal  with  these 
complexities ,  none  have  found  the  support  of  more  than  a 
small  group  of  individuals  or  organizations.  This  section 
will  address  the  complexities  involved  in  the  R&D  project 
selection  process  and  the  attempts  to  deal  with  them. 

It  must  be  first  noted  that  the  selection  of  research 
and  development  projects  is  a  subset  of  an  ever-changing 
innovation  process  that  involves  all  levels  of  an  organ- 
ization. One  model  of  this  so-called  research  management 
cycle   is  presented  in  Figure  1.    As  can  be  seen  from  this 
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diagram,  all  levels  of  the  organization  (labeled  on  the  left 
side  of  the  diagram)  are  involved  in  all  stages  of  the 
research  cycle  in  a  cyclical,  continuous  process.  This 
model  includes  idea  generation,  objective-setting,  planning, 
selecting,  and  controlling  R&D  projects.  Other  models  have 
been  proposed  by  Souder  [S-6]  and  Roberts  [R-2]. 

The  immediate  inclination  is  to  define  the  selection 
process  as  the  single  outlined  box  labeled,  "PORTFOLIO: 
PROJECT  SELECTION,  FUNDING."  More  realistically,  many 
other  elements  must  be  included  in  a  complete  selection 
model.  Models  vary  greatly  with  respect  to  the  range  of 
activities  that  are  modeled  into  the  selection  process. 
Many  consider  the  total  selection  process  to  include  the 
establishment  of  organizational  goals  and  the  evaluation  of 
projects,  as  well  as  the  actual  selection  decision. 

An  explanatory  definition  of  the  R&D  project  selection 
process  is  proposed  by  Baker  [B-l]: 


The  R&D  project  selection/resource  allocation 
problem  is  as  follows:  Given  a  set  of  alter- 
natives (projects  and  proposed  projects)  which 
require  common  scarce  resources  ( such  as  dollar 
budgets,  manpower,  and  facilities),  determine  that 
allocation  of  the  resources  to  the  alternatives 
which  will  maximize  the  benefit  contribution 
(value)  of  the  resulting  program. 


This  definition  outlines  the  R&D  project  selection  process 
as  determining  the  best  set  of  projects  for  the  organ- 
ization.  Although  this  may  be  an  accurate  definition  of  the 
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process,  the  actual  determination  of  what  is  "best"  for  the 
organization  is  a  very  complex  problem.  In  fact,  many 
complexities  of  the  R&D  project  selection  are  unique  and  are 
not  commonly  encountered  in  other  managerial  decisions. 

Discussions  of  these  unique  complexities  are  common  in 
literature  [B-5] [B-ll] [C-2] [G-4] [G-5] .  Table  2  summarizes 
these  discussions  with  a  list  of  ten  unique  complexities  of 
the  research  and  development  project  selection  process. 

The  first  factor  is  the  existence  of  ever-changing, 
fuzzy  organizational  goals.  The  process  of  extracting  the 
goals  of  an  organization  from  all  the  involved  individuals 
is  a  very  difficult  task.  It  is  difficult,  even  for 
individuals,  to  translate  these  often  abstract  goals  into 
concise  statements.  Furthermore,  these  goals  must  be 
translated  into  statements  upon  which  everyone  can  agree. 
Even  if  this  is  accomplished,  these  goals  can  change  quickly 
if  there  are  changes  in  either  the  internal  or  external 
environment. 

These  multiple  goals  are  commonly  of  a  non-quantifiable 
nature  and  non-commensurate.  Often  profit  is  quoted  as  the 
one  organizational  goal.  While  profit  is  often  included, 
other  goals  are  typically  listed  as  equally  important.  Such 
goals  might  include  diversification,  management  satisfac- 
tion, improved  public  image,  and  so  on.  Not  only  are  these 
qualitative  goals  very  difficult  to  measure,  but  they  are 
even  more  difficult  to  measure  on  a   common   scale.    This 
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TABLE  2:   Ten  Unique  Complexities  of  the  R&D  Project 
Selection  Process. 


1.  Multiple,  ever-changing,  fuzzy  organizational  goals. 

2.  Multiple  goals  are  qualitative  and  non-commensurate. 

3.  Various  levels  of  uncertainty. 

4.  Investments  are  often  large  and  vital  to  the  future  of 
the  organization. 

5.  Many  organizational  units  are  involved  whose  goals  are 
conflicting. 

6.  Process  is  sequential  and  continuous. 

7.  Projects  have  unequal,  variable,  and  uncertain  time 
horizons. 

8.  Involves  a  great  number  of  interdependent  alternatives. 

9.  Significant  dependence  on  outside  factors. 

10.   Need  to  motivate  and  retain  quality  R&D  personnel. 
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complicates   the   process   of   determining   the   trade-offs 
between  competing  goals  and  resource  limitations. 

The  third  complexity  is  the  inherent  uncertainty  in 
research  and  development.  Costello  [C-10]  states  that,  by 
definition,  research  involves  uncertainty: 


"...lack  of  information  about  the  future  potential 
of  research  ideas  is  one  of  the  features  that 
defines  research.  If  there  is  enough  information 
available  to  conduct  detailed  estimates  of  costs 
and  benefits,  the  project  can  no  longer  be 
considered  at  the  research  stage." 


Additionally,  all  project  selection  decisions  will  involve  a 
mix  of  R&D  projects  including  those  at  various  stages  of 
development.  Therefore,  these  projects  will  be  subject  to 
various  levels  of  uncertainty.  For  example,  the  rewards  of 
a  fundamental  research  project  are  virtually  unpredictable, 
while  the  returns  of  a  cost-reduction  effort  may  be  easily 
estimated.  These  various  levels  of  uncertainty  make  it 
difficult  to  make  direct  comparisons  between  projects. 

The  very  nature  of  research  and  development  projects 
are  that  they  deal  with  new  knowledge  or  new  applications. 
Consequently,  these  projects  usually  involve  new  products, 
processes,  or  services  for  the  organization.  Not  only  do 
these  projects  require  significant  investment  of  resources, 
but  the  outcome  of  the  projects  often  weighs  heavily  on  the 
future  ability  of  the  organization  to  provide  adequate 
products  and  services. 
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Additionally,  the  project  selection  process  often 
determines  the  utilization  of  the  major  resources  of  the 
organization  for  a  significant  length  of  time.  Once  these 
resource  allocations  have  been  made,  organizational  inertia 
makes  it  difficult  to  change  priorities  and  allocations. 
This  inertia  emphasizes  the  importance  of  initial  project 
selection  decision. 

As  noted  earlier,  research  and  development  involves 
almost  all  aspects  of  the  organization  from  accounting  to 
production.  Therefore,  each  of  these  units  must  be  involved 
to  provide  input  into  the  selection  process.  The  difficulty 
of  integrating  the  needs  of  these  diverse  organizational 
units  is  further  complicated  by  conflicts  in  goals.  For 
example,  while  finance  may  want  the  most  profitable  alter- 
native, the  R&D  department  may  desire  a  more  technologically 
advanced  alternative  while  production  may  insist  on  a  easily 
manufactured  alternative.  Different  levels  of  the  hierarchy 
of  the  organization  may  also  be  involved.  The  difficulties 
involved  in  the  resolution  of  these  conflicting  goals  is 
unique  to  the  R&D  selection  process  because  of  the  universal 
interest  in  these  projects  among  all  major  organizational 
units. 

The  typical  progression  from  idea  generation  to  commer- 
cialization of  an  R&D  project  is  continuous.   Yet,  decisions 
must  be  made  as  to  the  viability  of  a  project   at  various 
points  of  the  project's  life  cycle.   The  goals  and  the 
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relative  importance  of  those  goals  change  as  the  project 
progresses.  These  facts  complicate  the  evaluation  of  even  a 
single  project.  The  organization,  though,  must  be  contin- 
ually searching  for  new  ideas  and  initiating  new  projects. 
Therefore,  the  selection  process  must  take  into  account  the 
changing  nature  of  each  project  and  must  continually 
consider  the  addition  of  new  projects  to  the  pool. 

Normal  economic  evaluation  problems  involve  projects 
whose  lives,  if  not  equal,  are  comparable  in  length. 
Research  and  development  projects,  though,  may  have  lives 
that  vary  from  3  months  to  30  years.  More  often  than  not, 
the  time  to  completion  of  any  given  project  cannot  be  stated 
without  at  least  some  uncertainty.  The  difficulty  in 
comparing  the  value  of  the  outcomes  of  projects  with  such 
uncertain  and  different  time  horizons  make  the  necessary 
direct  comparisons  very  subjective  and  complex. 

If  an  organization  is  effective  at  idea  generation,  the 
number  of  possible  projects  is  endless.  These  projects  must 
first  be  screened  according  to  some  criteria  to  narrow  the 
field  to  a  reasonable  number  of  alternatives.  This 
screening  process  is  often  overly  subjective  and  good 
projects  can  easily  get  lost  in  the  shuffle.  Also,  the 
selection  of  many  projects  will  be  interdependent  upon  the 
acceptance  of  other  projects.  Either  the  projects  use  the 
same  resources  and  therefore  are  mutually  exclusive,  or 
projects  could  be  complementary  in  that  they  could  both 
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benefit  from  a  common  research  effort.  These  interrelation- 
ships are  very  difficult  to  incorporate  into  a  project 
selection  model. 

Since  R&D  projects  often  are  involved  with  the 
development  of  products,  processes,  or  services  that  will  be 
offered  to  the  customers  of  the  organization,  many  outside 
factors  can  determine  the  success  or  failure  of  the 
projects.  Competitive,  economic,  and  governmental  factors 
often  play  a  major  role  in  the  long-term  success  of  R&D 
projects.  These  factors  must  be  predicted  with  much 
accuracy  to  insure  the  correct  selection  of  projects. 

Finally,  many  organizations  must  make  a  concentrated 
effort  to  retain  certain  R&D  personnel.  Often  this  is  done 
by  providing  challenging  work  in  the  individual's  area  of 
expertise,  even  though  this  work  may  not  be  consistent  with 
other  objectives  of  the  organization.  The  need  to  motivate 
certain  personnel  must  also  be  considered  in  project 
selection  decisions. 

Baker  [B-l]  offers  an  excellent  discussion  of  the 
complications  involved  in  modeling  the  research  and 
development  project  selection  process.  He  offers  this 
summary  of  the  process: 


In  summary,  the  R&D  project  selection 
decision  is  a  process  by  which  an  intermittent 
stream  of  changes  are  made  to  lists  of  currently 
active  and  proposed  projects.  The  project 
selection  decision  process  includes  generating 
alternatives,   determining  the  appropriate  time  to 
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make  a  decision,   collecting  data,   specifying 
constraints  and  criteria,  and  recycling. 

The  evaluation  and  comparison  of  projects  and 
proposals  is  complicated  by  multiple  decision 
criteria  which  have  no  natural,  common  underlying 
measure  and  whose  relative  importance  varies  over 
time.  In  addition,  the  R&D  project  selection 
decision  may  be  made  at  several  different  organ- 
izational levels  which  are  participating  in  a 
hierarchical,  diffuse  budgeting  and  planning 
process. 


In  conclusion,  research  and  development  is  defined  as 
"the  discovery  and  application  of  new  knowledge  and  the  new 
applications  of  existing  knowledge."  The  R&D  selection 
process  is  complicated  by  many  organizational  and  external 
factors.  The  difficulty  of  modeling  all  of  these  complex- 
ities is  reflected  in  the  diverse  approaches  to  the  problem 
as  proposed  in  literature.  Chapter  Three  will  discuss  the 
different  proposed  approaches  to  the  R&D  project  selection 
decision  and  their  ability--or  inability--to  account  for  the 
above  described  complexities. 
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CHAPTER  THREE: 
R&D  PROJECT  SELECTION  MODELS 

Because  of  the  highly  complex  nature  of  the  R&D  project 
selection  decision,  the  topic  has  received  much  attention  in 
the  literature.  Most  of  the  articles  are  concentrated  in 
three  journals:  Research  Management,  IEEE  Transactions  on 
Engineering  Management,  and  Management  Science.  Although 
many  models  have  been  proposed  to  deal  with  the  various 
complexities  of  the  problem,  no  model  has  earned  the  support 
of  more  than  a  few  individuals  or  organizations. 

The  attention  to  the  problem  of  R&D  project  selection 
began  in  the  1950s  and  management  science  theory  has  since 
progressed  through  simple  scoring  models,  mathematical  pro- 
gramming models,  and  project  evaluation  techniques  to  the 
current  trend  in  modeling  using  organizational  decision 
methods  [S-14].  This  chapter  will  discuss  the  use  of  R&D 
project  selection  models,  the  current  trend  toward  group 
decision  making  methods,  and  a  new  model  proposed  by  Dr.  Wm. 
E.  Souder.  Chapter  Four  will  introduce  an  improved  project 
selection  system. 

Actual  Use  of  Models 

The  inability  of  currently  proposed  project  selection 
models   to  take  into  account  all  the  complexities   described 
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in  the  previous  chapter  is  evidenced  by  the  low  rate  of  use 
of  these  models.  The  model  makers  often  are  forced  to  make 
over-simplifying  assumptions  in  order  to  model  the  problem. 
On  the  other  hand,  other  model  makers  demand  so  much  input 
data  that  the  application  of  the  model  becomes  impossible. 
The  end  result  is  that  most  potential  users  ignore  the 
current  literature  and  continue  to  use  simple  economic 
evaluation  methods  or  no  organized  selection  process  at  all. 

Numerous  studies  have  been  conducted  in  the  United 
States  and  other  nations  to  determine  the  use  of  management 
science  models  for  R&D  project  selection  [C-l] [H-l] [L-4] [M- 
2]  [S-2]  [S-ll]  .  All  of  the  studies  have  come  to  the  same 
conclusion:  the  models  currently  proposed  in  the  literature 
are  not  being  utilized.  While  a  very  few  organizations  use 
mathematical  programming  techniques,  almost  all  organiza- 
tions, that  use  any  organized  method  at  all,  use  only  simple 
economic  evaluation  methods  such  as  internal  rate  of  return 
and  payback.  These  simplistic  evaluation  techniques  fail 
to  take  into  account  any  of  the  complexities  of  this  type  of 
decision.  Even  in  organizations  where  more  sophisticated 
models  have  been  introduced,  the  use  of  the  model  is  most 
often  discontinued  as  soon  as  the  developer  of  the  model 
leaves  the  organization. 

Dr.  Wm.  E.  Souder,  a  Professor  of  Industrial  Engineer- 
ing at  the  University  of  Pittsburgh,  is  a  widely  recognized 
as  one  of  the  leaders  in  the  field  of  R&D  project  selection. 
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Souder  [s-10]  summarizes  some  of  the  reasons  why  management 
science  models  are  over-simplified  compared  to  the  real- 
world  environment.  Table  3  lists  the  assumptions  made  by 
typical  management  science  models  (left  column)  and  the 
respective  real-world  situation  (right  column).  The 
differences  are  sharp  and  help  explain  the  limited  use  of 
such  models . 

Evolution  of  R&D  Project  Selection  Theory 

Since  the  first  real  emphasis  on  R&D  project  selection, 
the  approach  of  the  management  science  modelers  has  changed 
in  four  stages.  The  simple  classical  models  of  the  1950s 
gave  way  to  much  more  sophisticated  mathematical  programming 
models  in  the  1960s.  In  the  1970s,  modelers  introduced 
various  types  of  project  evaluation  techniques.  Currently, 
the  trend  is  toward  group  decision  making  models,  or 
organizational  decision  methods. 

This  division  of  the  progression  or  evolution  of  R&D 
project  selection  models  into  four  stages  was  proposed  by 
W.E.  Souder  and  T.  Mandakovic  in  an  article  entitled  "R&D 
Project  Selection  Models"  published  in  1986  [S-14].  This 
article  discusses  the  progression  of  management  science 
theory  and  the  weaknesses  and  strengths  of  many  represent- 
ative models.  Much  of  the  following  discussion  is  taken 
from  this  article. 
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TABLE  3: 


The  View  of  Management  Science  Models  vs. 
The  Real  World  Environment.  [S-10] 


The  View  of  Management 
Science  Models 


The  Real  World  Environment 


1.  A  single  decision  maker, 
in  a  well-behaved  environ- 
ment. 

2.  Perfect  information  about 
candidate  projects;  outputs, 
values  and  risks  of  candi- 
dates known  and 
quantifiable. 

3.  Known,  invariant  goals. 

4.  Decision  making  infor- 
mation is  concentrated  in 
the  hands  of  the  decision 
maker,  so  that  he  has  all 
information  he  needs  to 
make  the  decision. 

5.  The  decision  maker  is 
able  to  articulate  all 
consequences. 

6.  Candidate  projects  are 
viewed  as  independent,  to  be 
individually  evaluated  on 
their  own  merits. 


1 .  Many  decision  makers  and 
influencers,  in  a  dynamic 
organizational  environment. 

2 .  Imperfect  information  about 
candidate  projects;  outputs 
and  project  values  difficult 
to  specify;  uncertainty 
accompanies  all  estimates. 

3.  Ever-changing,  fuzzy  goals. 

4.  Decision  making  information 
is  highly  splintered  and  scat- 
tered throughout  the  organ- 
ization, with  no  one  part 
having  all  the  information 
needed  for  decision  making. 

5.  The  decision  maker  is  often 
unable  or  unwilling  to  state 
outcomes  and  consequences. 

6.  Candidate  projects  are 
often  technically  and  econom- 
ically interdependent. 


7.  A  single  objective, 
usually  expected  value  or 
profit  maximization,  is  ass- 
umed and  the  constraints  are 
primarily  budgetary. 

8.  The  best  portfolio  of 
projects  is  determined  on 
economic  grounds . 

9.  The  budget  is  "optim- 
ized" in  a  single  decision. 

10. One  single,  economic- 
ally "best" ,  overall 
decision  is  sought. 


7 .  There  are  sometimes  con- 
flicting multiple  objectives 
and  multiple  constraints,  and 
these  are  often  non-economic 
in  nature. 

8.  Satisfactory  portfolios 
may  possess  many  non-economic 
characteristics . 

9.  An  iterative,   re-cycling 
budget  process  is  used. 

10.  The  "best"  decision  for 
the  total  organization  may  not 
be  "best"  for  each  dept. 
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The  classical  models  were  developed  to  prioritize 
projects  in  order  to  choose  the  "best"  project.  Such 
typical  models  included  profiles,  checklists,  scoring,  and 
economic  indexes.  Profiles  simply  display  the  decision 
maker's  preferences  on  a  graph.  Checklists,  on  the  other 
hand,  solicit  a  score  from  the  decision  maker  on  a  number  of 
selection  criteria.  The  total  score  for  each  project  is 
simply  the  sum  of  the  individual  scores  for  each  criterion. 

Many  of  the  shortcomings  of  these  models  were  overcome 
with  the  use  of  scoring  models.  Like  checklists,  scores  are 
solicited  for  each  of  the  selection  criteria.  The  total 
score  for  each  project  using  a  scoring  model  is  a  weighted 
sum  of  the  scores  of  the  criteria  based  on  the  relative 
importance  of  the  criteria.  Even  with  this  improvement,  the 
model  remains  overly  subjective  because  of  the  intuitive 
manner  in  which  both  the  weights  and  the  scores  are  derived. 

As  noted  earlier,  economic  indexes  are  one  of  the  most 
common  types  of  project  selection  methods  used  today.  Yet, 
these  models  rely  heavily  on  difficult  estimates  of  the 
probability  of  success  of  each  project.  These  models  also 
disregard  most  non-economic  factors  involved  in  the 
selection  process. 

The  1960s  saw  an  increase  in  the  development  of 
portfolio  selection  models  based  on  mathematical  program- 
ming. The  first  models  used  a  linear  programming  approach. 
These  models  were  later  improved  by  the  introduction  of  non- 
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linear,  integer,  and  dynamic  programming  models.  These 
sophisticated  models  allowed  decision  makers  to  incorporate 
many  of  the  resource  allocation  constraints  into  the  model 
of  the  problem.  The  biggest  limitation  of  these  models  was 
the  excessive  input  data  requirements.  Further,  because  of 
their  complexity,  many  decision  makers  could  not  understand 
the  sometimes  overwhelming  mathematics  and,  therefore, 
rejected  the  models  altogether. 

A  number  of  different  types  of  project  evaluation  tech- 
niques have  been  developed  and  applied  to  the  R&D  project 
selection  problem  during  the  1970s.  These  models  include 
goal-  or  value-contribution  models,  decision  trees,  utility 
theory  models,  Monte  Carlo  simulation,  and  risk  analysis. 
The  models  are  helpful  to  the  decision  maker  in  evaluating 
individual  projects  or  in  understanding  the  multiple 
selection  criteria  involved  in  the  selection  process.  Many 
of  these  methods  are  still  being  proposed  and  tested  in  the 
field. 

The  most  recent  trend  in  the  modeling  of  the  R&D 
selection  process  is  the  movement  toward  models  that 
incorporate  the  role  and  behavior  of  the  many  decision 
makers  in  an  organizational  setting.  Souder  identifies  two 
such  approaches:  behavioral  decision  aids  and  decentralized 
hierarchical  modeling.  Behavioral  decision  aids  include 
many  group  decision  making  methods  such  as  the  Delphi 
Method,   Nominal  Group  Techniques,   Q-Sorting,   nominal,  and 
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interactive  group  settings.  These  methods  have  two  aims: 
1)  to  assemble  the  necessary  information  from  the 
individuals  of  the  organization  to  make  a  project  selection 
decision  (using  an  existing  or  original  project  selection 
method),  and  2)  by  collecting  this  information  in  a  group 
decision  making  environment,  to  improve  communication  and 
motivation  in  the  organization. 

Decentralized  hierarchical  modeling  is  a  method 
designed  to  integrate  the  various  organizational  units  in  a 
hierarchical  organization.  The  method  utilizes  computer 
terminals  to  collect  input  from  all  the  units  involved  in 
the  resource  allocation  process.  Once  the  input  is 
collected,  a  suggested  portfolio  is  developed  and  the 
proposal  is  submitted  to  each  unit  for  evaluation  [M-3][B- 
4].  This  process  is  repeated  until  a  consensus  is  reached. 
Similar  hierarchical  models  have  been  proposed  based  on 
group  decision  making  methods  [L-5]. 

The  evolution  of  R&D  project  selection  models  has 
progressed  from  simplistic  models  that  failed  to  account  for 
the  complexities  of  the  problem  to  the  overly-complex 
mathematical  models  that  ignored  the  limitations  of  the 
decision  maker.  Only  recently  has  adequate  attention  been 
given  to  the  needs  and  limitations  of  the  decision  makers 
themselves  [S-13],  Only  with  the  input  and  participation  of 
all  the  key  individuals  and  groups  within  an  organization 
will   the    "optimum"    portfolio  of   projects    for   the 
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organization  be  found. 

Therefore,  a  need  exists  for  the  development  of 
complete  project  selection  systems  integrating  the 
collection  of  data  and  the  solicitation  of  goals  from  the 
decision  makers  and  the  use  of  this  information  to  select  a 
portfolio  of  research  and  development  projects.  The  next 
section  will  discuss  one  such  system  proposed  by  Souder  in 
1978  and  Chapter  Four  will  be  devoted  to  the  introduction  of 
a  proposed  selection  system  based  on  the  classification  of 
projects. 

Souder 's  Project  Evaluation  System 

This  proposed  system  is  based,  to  a  large  extent,  on 
a  Q-Sorting/Nominal-Interacting  (QS/NI)  process  which  has 
been  tested  in  the  field  for  use  in  the  R&D  project 
selection  process  [S-5][S-6].  The  use  of  this  process  is 
combined  with  the  use  of  more  traditional  project  selection 
models  depending  on  the  type  of  projects  under  consideration 
and  the  type  of  decision  involved. 

The  project  selection  process  itself  is  modeled  in 
Figure  2.  This  simple  flow-chart  outlines  the  steps  and 
thought  processes  involved  in  the  project  selection 
decision.  Beginning  with  the  alternative  project  ideas,  the 
ideas  are  screened,  evaluated,  prioritized,  and  finally 
combined   into   an   "optimum"   portfolio.    The  necessary 
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FIGURE  2:   Souder ' s  Project  Selection  Process.  [S-7] 


30 


feedback  loops  are  included  for  the  abandonment  of  ideas  or 
the  addition  of  the  rejected  projects  to  the  organization's 
inventory  of  potential  project  ideas.  The  final  output  of 
the  process  is  the  organization's  R&D  portfolio. 

Figure  3  presents  a  system  for  using  project  selection 
techniques  also  proposed  by  Souder.  The  integration  of 
these  two  system  presents  a  complete  project  selection  and 
evaluation  system.  By  following  the  selection  system  in 
Figure  2  and  using  Figure  3  as  a  guide  at  each  stage  of  the 
selection  process,  a  total  evaluation  system  emerges. 

The  first  step  in  any  decision  process,  as  modeled  in 
Figure  3,  is  the  establishment  of  a  consensus  set  of  goals. 
This  necessary  step  must  precede  the  use  of  any  project 
selection  method.  The  rest  of  the  model  gives  guidelines 
for  the  application  of  R&D  project  selection  methods 
according  to  the  type  of  projects  being  compared  and  by  the 
type  of  decision  being  made. 

The  top  row  divides  the  projects  to  be  compared  by 
project  type,  or  life  cycle  stage.  Exploratory  projects  are 
characterized  by  the  lack  of  evaluative  information  and 
consequently  require  models  capable  of  handling  more 
subjective  input  data:  checklists  or  profiles.  For  applied 
projects,  usually  some  evaluative  information  is  available. 
For  these  projects,  Souder  recommends  the  use  of  Q-Sorting 
to  screen  the  projects  followed  by  the  use  of  an  index, 
scoring,   risk   analysis,   or  frontier  model  for   the   final 
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FIGURE  3: 


Souder's  Guidelines  for  the  Use  of  R&D  Project 
Selection  Methods.  [S-10] 
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selection.  The  availability  of  a  greater  quantity  of  less 
subjective  data  on  development  projects  allows  the  decision 
maker  to  apply  more  sophisticated  computerized  resource 
allocation  models.  The  input  for  the  model  could  be  derived 
using  a  Nominal-Interacting  process. 

The  second  row  of  Figure  3  divides  the  use  of  project 
selection  methods  by  the  decision  type.  These  decision 
types  correspond  to  the  selection  stages  presented  in  Figure 
2.  As  the  decision  maker  progresses  through  the  screening, 
prioritizing,  and  resource  allocation  stages  of  the 
selection  process,  he/she  may  use  the  guidelines  in  Figure 
3  to  choose  the  appropriate  decision  method.  For  screening 
decisions,  only  subjective  accept/reject  decisions  are 
necessary.  Almost  any  model  can  be  used  for  this  stage 
depending  upon  the  form  of  the  available  input  information. 
For  prioritizing  decisions,  more  evaluative  information  is 
usually  available  and  necessary.  Therefore,  more  sophisti- 
cated models  could  be  used  for  prioritization.  Finally,  as 
the  selection  process  reaches  the  portfolio  selection  stage, 
much  information  has  been  collected  on  each  of  the  remaining 
projects  and  the  use  of  a  portfolio  model  is  possible  and 
justified. 

Thus,  this  total  selection  process  consists  of  three 
selection  stages  and  guidelines  for  carrying  out  the 
decisions  involved  at  each  of  these  stages.  The  QS/NI 
process   is  used  with  other  models  at  each  stage  to  solicit 
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the  input  of  the  decision  makers. 

In  conclusion,  the  inadequacy  of  R&D  project  selection 
models  as  proposed  in  literature  has  been  evidenced  by  the 
lack  of  use  of  such  methods  in  real  world  settings.  Often 
too  many  simplifying  assumptions  are  made  or ,  on  the  other 
extreme,  the  input  requirements  of  the  model  are  overly 
demanding  to  make  the  use  of  the  model  practical.  The 
solution  to  the  organizational  complexities  of  this  decision 
process  is  the  coupling  of  group  decision  making  methods 
with  the  use  of  these  selection  models.  One  such  system 
proposed  by  Souder  was  presented.  This  system  outlined  the 
selection  process  and  provided  guidance  to  the  decision 
maker  as  to  the  use  of  selection  methods  at  the  various 
stages  of  the  process  and  for  different  project  types. 

The  following  chapter  will  present  a  proposed  system  for 
incorporating  the  decision  makers  in  the  organization  and 
increasing  the  effectiveness  of  R&D  project  selection 
methods.  This  system  is  based  on  the  classification  of  R&D 
projects.  This  classification  of  R&D  projects  will  allow 
many  existing  models  to  be  used  much  more  effectively. 
Additionally,  the  use  of  group  decision  making  methods  will 
improve  the  quality  of  the  input  data  into  such  methods  and 
will  benefit  the  organization  in  other  important  ways. 
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CHAPTER  FOUR: 
A  PROPOSED  R&D  PROJECT  SELECTION  SYSTEM 

The  selection  of  research  and  development  projects  from 
an  often  exhaustive  list  of  project  proposals  is  a  uniquely 
difficult  managerial  decision.  The  size  of  the  investment, 
the  importance  of  the  projects  to  the  continued  operation  of 
the  organization,  the  involvement  of  nearly  every  component 
of  the  organizational  structure,  and  the  substantial 
uncertainty  in  predicting  possible  outcomes  make  this 
decision  process  uniquely  complex.  Consequently,  much 
attention  has  been  given  to  the  problem,  yet  few  viable 
selection  systems  have  been  proposed  and  applied. 

The  complex  mathematical  programming  methods  have  suc- 
ceeded in  many  aspects  in  incorporating  many  of  the  com- 
plexities of  this  decision.  Yet,  as  the  models  incorporate 
more  of  the  complexities,  the  data  requirements  for  the 
models  grow  proportionately.  Since  much  of  this  data  is  of 
a  non-quantitative  nature  and/or  very  difficult  to  estimate, 
the  models  soon  become  impractical.  For  example,  many 
models  require  estimates  of  return  per  dollar  spent,  prob- 
abilities of  technical  and  commercial  success,  time  to 
completion,  market  demand  and  competitive  factors,  and 
complex  tradeoffs  between  multiple  objectives  and  resource 
constraints.    Additionally,   no  one  decision  maker  has   all 
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the  information  necessary  to  make  these  tradeoffs,  so  the 
decision  must  be  a  consensus  of  numerous  individuals  who 
have  conflicting  goals,  scattered  information,  and  diverse 
points  of  view  on  the  total  process. 

As  discussed  in  the  previous  section,  the  only  way  to 
provide  such  stringent  data  requirements  is  to  provide  a 
structured  environment  in  which  all  the  key  individuals  in 
the  R&D  process  can  exchange  information  and  work  towards  a 
consensus.  Once  the  input  requirements  are  fulfilled,  an 
appropriate  model  should  be  formulated  and  solved.  These 
results  are,  in  turn,  presented  to  the  group  for  further 
consideration.  This  method  achieves  the  analytical 
advantages  of  using  a  selection  model  and  the  organizational 
advantages  associated  with  group  decision  making:  improved 
communication,  better  cooperation,  and  intrinsically 
motivated  managers. 

One  such  model  was  proposed  by  Souder  and  was  presented 
in  the  preceding  chapter.  This  chapter  will  introduce  a 
project  selection  system  developed  by  the  author  to  provide 
the  basis  for  the  formulation  of  goals  and  the  practical 
application  of  project  selection  methods. 

This  new  system  shares  some  characteristics  with  the 
"convergent  approach"  to  R&D  project  selection  proposed  by 
Clayton  [C-4].  Clayton  recommended  an  approach  based  on  the 
theory  that  top  management  should  be  responsible  for 
establishing  broad  organizational  goals  while  the  individual 
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R&D  units  should  determine  how  these  broad  goals  might   be 
best  accomplished. 

The  proposed  system  is  represented  in  Figure  1.  The 
system  consists  of  four  phases:  I)  establishing  organi- 
zational goals,  II)  classifying  the  R&D  projects  into 
groups.  III)  choosing  or  ranking  the  projects  within  each 
class,  and  IV)  pooling  the  projects  from  all  the  classes  and 
selecting  a  portfolio.  The  establishment  of  organizational 
goals  and  the  selection  of  the  portfolio  (Phases  I  and  IV) 
are  carried  out  by  a  committee  consisting  of  all  of  the  key 
individuals  in  the  R&D  process.  The  classification  of 
projects  and  the  choosing/ranking  of  projects  within  each 
class  (Phases  II  and  III)  is  done  by  the  individual  R&D 
units  or  a  group  of  R&D  managers. 

Phase  I:   Establish  Organizational  Goals 

The  formulation  of  a  set  of  organizational  goals  is  a 
prerequisite  for  any  viable  project  selection  system. 
Without  some  agreement  and  subsequent  recording  of  the  goals 
of  the  organization,  the  R&D  process  cannot  be  effectively 
directed  to  further  these  goals.  Since  the  R&D  project 
selection  process  is  the  most  direct  means  of  establishing 
the  current  and  future  orientation  of  an  organization's  R&D 
effort,  these  goals  are  critical  to  the  project  selection 
decision. 
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FIGURE  4:   A  Proposed  R&D  Project  Selection  System. 
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Table  4  lists  a  number  of  references  for  discussions 
and  surveys  of  organizational  goals  and  R&D  project 
selection  criteria.  This  table  lists  the  reference  number, 
author  (or  authors),  date,  and  a  brief  description  of  the 
contents  of  each  article.  The  recommendations  of  different 
researchers  on  project  selection  criteria  vary  greatly. 
Souder  [S-7]  groups  all  criteria  into  four  categories: 
costs,  benefits,  risks,  and  suitability.  Cooper  [C-7] 
groups  criteria  into  the  more  long-range  oriented  categories 
of  impact,  feasibility,  and  intrinsic  scientific  merit.  Of 
course,  each  organization  must  establish  its  own,  unique  set 
of  goals.  These  references  are  intended  to  provide  a  list 
of  possible  goals  to  provide  a  starting  point  for  decision 
makers  formulating  their  own  goal  structures . 

The  individuals  involved  in  the  decision  making  process 
must  first  organize  their  own  understanding  of  the  selection 
process  and  criteria  before  they  will  be  able  to  provide 
useful  input  to  the  group  decision  making  effort  [S-6]. 
Opportunities  for  individuals  to  determine  their  own  goal 
structures  must  be  provided  in  the  total  group  decision 
making  process.  Once  these  individual  goals  are  formulated, 
a  consensus  must  be  formed  from  the  group  on  statements  of 
the  goals  of  the  organization. 

Since  these  goals  must  provide  the  basis  for  the 
formulation  of  project  selection  criteria,  the  statement  of 
these   goals  must  be  clear  and  concise.    A   set   of   vague 
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TABLE  4:  References  for  the  Establishment  of  Organizational 
Goals/R&D  Criteria. 


Ref.   Author (s)     Date 
B-7   Becker        1980 


Description 


C-7 

Cooper 

1978 

C-8 

Cooper 

1980 

C-9 

Cooper 

1981 

F-2 

Fernelius 

1979 

M-5 

Merrif ield 

1981 

R-l 

Ramsey 

1978 

S-2 

Schwartz 

1977 

S-3 

Souder 

1975 

S-7 


Souder 


1984 


IRI  survey  of  criteria  for 
research,  product  &  process 
development. 

Discussion  of  impact,  feasibility, 
and  intrinsic  scientific  merit. 

Survey  of  new  product  selection 
criteria. 

Extension  of  C-8. 

IRI  survey  on  innovative  process. 

Criteria  for  commercial  success. 

Book:   "Research  and  Development 
Project  Selection  Criteria." 

Survey  of  Canadian  top  executives . 

Presents  a  method  of  achieving 
organizational  consensus  on  R&D 
project  selection  criteria. 

Criteria  by  costs,  benefits, 
risks,  and  suitability. 
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statements  such  as  "improve  the  stability  of  the  company" 
without  specifying  how  this  "stability"  is  to  be  measured, 
offers  little  assistance  to  model  makers.  Possible  measures 
for  the  suggested  goals  should  always  be  considered. 
Finally,  some  feeling  for  the  relative  importance  of  all  the 
enumerated  goals  must  be  established.  At  a  minimum,  the 
goals  should  be  rank-ordered  as  to  their  importance. 

Numerous  group  decision  making  methods  have  been 
devised  to  assist  in  forming  a  consensus  from  a  group  of 
individuals.  Several  have  been  field  tested  in  the  R&D 
project  selection  environment  and  the  results  have  been 
reported  in  the  literature.  The  references  to  these  studies 
are  provided  in  Table  5.  An  article  by  Salasin,  et  al.  [S- 
1],  compares  the  Nominal  Group  Technique  (NGT)  and  the 
Delphi  method  with  respect  to  the  amount  and  quality  of  the 
data,  cost,  and  the  participant's  satisfaction  with  the 
process.  Souder  [S-6]  reports  the  test  of  the  Q- 
Sorting/Nominal-Group  Process  in  two  R&D  organizations. 

These  group  techniques  should  be  used  with  a  committee 
of  all  the  key  individuals  involved  in  the  R&D  process.  The 
composition  of  this  committee  will  vary  greatly  according 
to  the  size,  structure,  and  type  of  organization.  The 
committee  could  range  from  a  meeting  of  the  company 
president  and  the  R&D  manager  to  a  committee  composed  of  20 
to  30  individuals  from  various  components  of  the 
organization.   Further  discussion  of  these  organizational 
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TABLE  5: 


Group  Decision  Making  Techniques  and  Applications 
in  R&D. 


Ref.   Author! s)      Date  Description 


D-l   Davies  &       1980 
Pearson 


Example  and  advantages  of  using 
group  decision  making/selection 
model  combinations. 


D-3   DeSanctis  &    1987 
Gallupe 


H-2    Hwang  &  Lin     1987 


K-2   Khorramshagfol  1986 
&  Gousty 

P-2   Plebani  &      1981 
Jain 


S-l   Salasin, 

Entingh,  & 
Thackston 


1981 


Future  directions  and  design 
factors  for  group  decision 
support  systems. 

Comprehensive  and  detailed  review 
of  group  decision  making  under 
multiple  criteria. 

Delphic  Goal  Programming  method 
is  proposed. 

A  Simple  Additive  Weighting  model 
using  NGT  and  Delphi  is 
presented. 

NGT  and  Delphi  are  compared  on 
several  criteria. 


S-5   Souder 


1977   Nominal,  interacting,  and 

combined  nominal-interacting  are 
compared  for  integrating  R&D  and 
marketing. 


S-6    Souder 


S-10   Souder 


S-15   Stahl  & 
Harrell 


v-1   Van  DeVen  & 
Delbecq 


1975  Tests  a  Q-Sorting/Nominal  Group 
Process  in  two  organizations. 

1978  Integrates  a  Q-Sorting/Nominal- 
Interacting  process  into  a  total 
project  selection  system. 

1983  Behavioral  Decision  Theory  is 
applied  to  identify  operative 
goals  in  an  Air  Force  R&D  lab. 

1974  Comparison  of  NGT,  Delphi,  and 
interacting  groups. 
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factors  can  be  found  in  Chapter  Six. 

Once  the  organizational  goals  have  been  established  and 
recorded,  the  committee  must  decide  on  a  classification 
scheme  for  their  unigue  set  of  R&D  projects.  There  are 
several  reasons  why  R&D  projects  should  be  classified  before 
any  selection  decisions  are  made.  First,  by  grouping 
together  projects  that  share  a  number  of  important 
characteristics,  better  comparisons  can  be  made  between 
projects.  Second,  by  allocating  a  certain  proportion  of  the 
total  R&D  budget  to  the  various  classes  of  projects,  the 
organization  can  insure  a  more  "balanced"  portfolio  of  R&D 
projects.  A  more  detailed  discussion  of  the  classification 
of  R&D  projects  will  be  presented  in  the  next  section. 

The  committee  may  then  determine  the  specific  fractions 
of  the  forecasted  R&D  budget  to  be  allocated  to  the 
different  classes  of  projects.  These  non-binding  guide- 
lines, will  help  the  committee  develop  a  more  balanced  R&D 
portfolio  in  Phase  IV  of  the  selection  system. 

As  already  noted,  the  classification  of  R&D  projects 
can  assist  an  organization  in  pursuing  a  balanced  R&D 
effort.  More  importantly,  the  classification  of  R&D 
projects  is  necessary  for  the  effective  application  of  most 
R&D  project  selection  methods.  These  models  cannot 
incorporate  the  diverse  characteristics  of  all  types  of  R&D 
projects.  By  classifying  projects,  some  of  the  differences 
between  projects  are  removed  and  more  realistic  comparisons 
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can  be  made. 

Phase  II:   Classify  Projects 

Every  time  an  organization  is  faced  with  an  R&D  project 
selection  decision,  the  decision  makers  are  faced  with  a  set 
of  alternatives  including  a  vast  mix  of  various  types  of 
projects.  These  projects  vary  greatly  in  the  amount  of 
uncertainty  involved  and  even  the  criteria  for  the  measuring 
the  success  of  each  project.  Since  the  direct  comparison 
of  all  these  R&D  projects  is  impossible,  most  selection 
methods  are  designed  to  deal  with  only  a  single  project 
type--often  development  projects.  The  classification  of 
projects  into  similar  project  types  will  reduce  the  number 
of  differences  between  projects  and  these  methods  can  be 
applied  within  the  confines  for  which  they  were  designed. 

Of  course,  the  classification  scheme  used  to  make 
allocations  in  Phase  I  should  be  the  same  scheme  used  for 
project  selection.  For  example,  if  the  coordinating 
committee  feels  it  needs  to  insure  a  balance  of  projects 
according  to  the  market  to  which  the  project  is  targeted, 
this  classification  scheme  should  also  be  used  for  project 
selection. 

The  classification  scheme  selected  by  the  committee  and 
used  in  the  allocation  of  the  R&D  budget  is  based  on  a 
characteristic   of   the  projects  which  is  considered   to  be 
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uniquely  important  to  the  organization.  This  project 
characteristic  might  be  the  level  of  risk  of  various 
projects,  the  newness  to  the  firm,  the  use  of  key  resources, 
completion  time,  etc.  Since  this  project  characteristic  has 
been  defined  as  being  important  to  the  organization,  it  is 
only  appropriate  to  use  this  same  classification  scheme  for 
project  selection.  The  result  is  that  only  projects  which 
share  this  crucial  characteristic  will  be  compared  directly 
[D-l]. 

Many  organizations  place  significant  emphasis  on 
retaining  a  "balanced"  R&D  effort.  The  purpose  of  this 
balancing  is  to  spread  the  risk  of  project  failure  over 
several  types  of  R&D  projects,  i.e.  to  diversify.  For 
example,  an  organization  may  desire  to  allocate  a  fixed 
percent  of  total  R&D  expenditures  to  basic  research  versus 
applied  and  development  efforts.  Companies  may  also  want  to 
spread  their  effort  over  projects  involving  existing 
products  and  processes  versus  new  lines  of  business.  Other 
committees  may  want  to  balance  completion  times,  new  vs. 
existing  markets,  risks,  long-run  vs.  short-run  profits,  or 
the  utilization  of  key  resources.  The  selection  of  an 
appropriate  classification  scheme  is  the  topic  of  Chapter 
Five. 

Some  typical  classification  schemes  for  research  and 
development  projects  include:  1)  stages  of  the  project's 
life   cycle,   2)   new  versus  existing  business,   3)   product 
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versus  process,  and  various  combinations  of  these  schemes. 
These  above  schemes  are  frequently  used  and  are  the  focus  of 
Chapter  Five. 

Phase  II  divided  the  mix  of  R&D  projects  into  classes 
according  to  the  goals  of  the  organization  and  the  unique 
characteristics  of  the  alternative  projects.  This  step 
groups  projects  together  that  can  be  evaluated  using  one  of 
the  available  R&D  project  selection  methods.  The  third 
phase  of  the  process  is  to  use  one  of  these  methods  to 
choose  among  or  rank  the  projects  in  each  class. 

Phase  III;   Choose/Rank  Projects  Within  Each  Class 

The  first  two  phases  of  the  project  selection  system 
have  established  the  organizational  goals  (and  therefore  the 
project  selection  criteria)  and  classified  the  alternatives 
into  groups  of  comparable  projects.  These  two  important 
steps  have  provided  the  necessary  preparation  for  the 
appropriate  use  of  many  of  the  currently  available  R&D 
project  selection  methods.  These  preparatory  steps,  are 
necessary  to  conform  to  the  restrictions  of  selection  models 
and  to  insure  the  appropriateness  of  the  results  to  the 
organization. 

Many  journal  articles  and  books  have  reviewed  a  number 
of  the  R&D  project  selection  methods  presented  in  the 
literature  and  have  analyzed  the  strengths,   weaknesses,  and 
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applications  of  each.  Table  6  presents  some  of  the  more 
comprehensive  discussions  in  this  area.  A  review  of  several 
of  these  articles  and  books  will  quickly  introduce  the 
decision  maker  to  the  range  of  models  available  and  the 
characteristics  of  each. 

Selecting  the  correct  selection  method  or  model  is  an 
important  and  difficult  decision.  No  one  method  is 
appropriate  for  all  cases.  Some  of  the  factors  to  be 
considered  when  choosing  a  model  include:  the  nature  of  the 
selection  criteria,  the  certainty  in  the  estimates  of  the 
probabilities  of  success  and  expected  returns,  and  the  class 
of  projects  being  evaluated. 

Some  models  require  quantitative  data  for  the 
measurement  of  the  selection  criteria.  Other  models  have 
the  ability  to  analyze  more  subjective  data  such  as 
subjective  ratings  (say  on  a  arbitrary  scale  from  1  to  10). 
Scoring  and  checklist  models  are  simple  models  to  use  with 
subjective  ratings  data,  especially  since  the  criteria  and 
the  relative  weight  among  the  criteria  have  already  been 
determined  in  Phase  I.  If  only  expected  return  is  to  be 
considered,  simple  decision  theory/expected  value  models  can 
be  used. 

Most  risk  analysis  and  frontier  models  require  a  range 
of  estimates  of  the  probability  of  technical  and/or  com- 
mercial success  and  the  expected  return  of  each  project.  If 
this   information   is   obtainable   and   risk   is   the   main 
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TABLE  6 :   Reviews 

Ref.   Author (s) 

A- 2   Augood 

B-2   Baker  & 
Freeland 

B-3   Baker  & 
Pound 

B-6   Balderston 


B-9    Booker  & 
Bryson 

C-l   Cetron, 

Martino,  & 
Roepcke 

C-3    Clarke 


D-2   Dean  & 

Nishry 

J-l   Jackson 


0-1   Office  of 
Technology 
Assessment 

S-7   Souder 


S-8    Souder 


S-10   Souder 


S-14   Souder  & 

Mandakovic 


of  R&D  Project  Selection  Models. 

Date   Description 

1973   Reviews  checklists  and  indices. 

1975   Discusses  benefit  measurement 
&  resource  allocation  models. 

1964  Well-quoted  review  of  10  models 
and  detailed  analysis  of  3 
representative  models. 

1984  Book:   gives  strengths   & 
weaknesses  of  2  types  of  models. 

1985  A  recent  and  complete  review  of 
most  types  of  models . 

1967   Well-quoted,  descriptive  review 
of  30  models. 


1974   Literature  survey  from  1967-73 
on  the  actual  use  of  models. 

1965   Comparison  of  scoring  and 
profitability  models. 

1983  Descriptions  &  examples  of  most 
types  of  models. 

1986  Broad  OTA  study  on  government  & 
industry  research  evaluation. 


1984   Book:   brief  discussion  of  most 
types  of  models . 

1972   26  models  compared  for  ease  of 
use,  realism,  flexibility, 
capability,  and  cost. 

1978   Describes  &  gives  examples  for 

8  types  of  models  and  recommends 
when  they  should  be  used. 

1986   Recent  discussion  of  historical 
evolution  of  R&D  selection 
methodology  and  models . 
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criterion  for  selection,  the  problem  should  be  solved  using 
one  of  these  models.  Conversely,  if  the  projects  have  been 
classified  by  the  level  of  risk,  each  project  within  the 
class  will  have  a  similar  risk  profile  and  other  criteria 
should  be  considered.  Single  estimates  of  the  probability 
of  success  can  be  incorporated  into  scoring,  index,  and 
decision  theory  models. 

Clearly,  the  type  of  projects  being  evaluated  effects 
the  type  of  models  to  be  used.  This  is  one  of  the  main 
arguments  for  classifying  the  projects  before  selecting. 
Most  of  the  differences  in  project  types  will  be  accounted 
for  by  the  two  factors  previously  discussed:  the  nature  of 
the  criteria  and  the  certainty  of  the  estimates  of  the 
probability  of  success  and  expected  returns.  These  factors 
are  greatly  determined  by  the  project  type. 

For  example,  say  the  projects  were  classified  into 
basic  research,  applied  research,  and  development  projects. 
The  basic  research  projects  would  be  characterized  by 
subjective  data  and  great  uncertainty  in  the  estimates  of 
the  probability  of  success  and  expected  returns.  Only 
simple  models  with  very  limited  data  requirements  could  be 
used  here.  Some  quantitative  data  is  typically  available 
for  a  project  in  the  applied  research  stage  yet  the  goals 
are  still  non-quantitative.  The  use  of  a  risk  analysis  or 
scoring  model  may  be  appropriate.  Finally,  development 
projects   are   typically  evaluated  on   strictly   economic 
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criteria  since  the  costs  and  potential  returns  are  reason- 
ably well  known.  An  expected  return  or  mathematical  pro- 
gramming method  could  be  used. 

If  allocations  were  established  for  each  class  of 
projects  in  Phase  I,  these  allocations  could  be  modeled  as 
budget  constraints  within  each  class.  The  best  projects 
would  be  chosen  subject  to  the  amount  budgeted  to  the 
particular  class.  The  limitation  of  the  budget  for  each 
class  could  limit  the  number  of  projects  to  be  considered 
and,  therefore,  simplify  the  selection  process.  The  use  of 
this  constraint  could  also  evoke  the  discussion  of  possible 
trade-offs  between  projects  within  each  class  which  could  be 
valuable  information  for  the  coordinating  committee  in 
establishing  the  overall  portfolio  in  Phase  IV. 

Once  the  projects  in  each  class  have  been  selected  or 
ranked,  this  information  is  passed  back  up  to  the  coordina- 
ting committee.  This  committee  meets  again  to  establish  the 
final  R&D  portfolio.  A  group  decision  making  technique  must 
be  integrated  with  an  appropriate  portfolio  model  to  arrive 
at  a  consensus  optimum  solution. 

Phase  IV:   Pool  Projects  and  Select  Portfolio 

The  final  phase  of  the  project  selection  system  is  the 
selection  of  the  organization's  R&D  portfolio.  In  many 
aspects,   an  R&D  portfolio  is  much  like  a   financial 
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portfolio.  The  correct  components  of  the  portfolio  must  be 
combined  in  order  to  maximize  the  return  and  minimize  the 
risk  to  the  organization.  While  a  balanced  (diversified) 
financial  portfolio  reduces  the  risk  of  the  investment,  a 
properly  balanced  R&D  portfolio  will  also  reduce  the  risk  of 
project  failure  to  the  organization. 

All  the  information  from  the  evaluations  conducted  in 
the  first  three  phases  of  the  system  are  designed  to  provide 
all  the  necessary  input  for  the  selection  of  an  optimum  R&D 
portfolio.  The  allocations  made  in  Phase  I  and  selection 
decisions  made  in  Phase  III  can  be  accepted  as  given  or 
modified  by  the  committee.  Strictly  speaking,  if  the 
allocations  for  each  class  were  filled  by  the  best  projects 
in  each  class,  the  "optimum"  portfolio  would  already  have 
been  determined.  Most  likely,  though,  some  changes  will  be 
made  in  both  the  allocations  and  the  selection  decisions 
within  each  class  to  maximize  the  total  return  on  the  port- 
folio (in  the  opinion  of  the  committee). 

Again,  it  will  be  necessary  to  structure  the  discussion 
using  some  group  decision  making  technique.  (See  Table  5.) 
Much  information  will  be  available  to  the  committee 
including  its  own  set  of  goals  and  possible  allocations  of 
the  budget  among  the  various  classes  of  projects  and  the 
project  selection  decisions  for  each  class  of  projects.  In 
order  to  structure  and  stimulate  the  discussion,  a  portfolio 
selection  method  should  be  formulated  to  compile  a  suggested 
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portfolio. 

The  use  of  a  portfolio  selection  model  will  give  the 
decision  makers  an  analytical  tool  for  making  trade-offs 
between  alternative  projects  and  funding  levels.  The  model 
can  be  used  to  answer  many  "what  if"  types  of  questions  such 
as:  "What  if  we  increase  the  funding  for  project  A  and 
decrease  the  funding  for  project  B?"  This  type  of  sensi- 
tivity analysis  can  be  performed  iteratively  until  the 
committee  comes  to  a  consensus  on  an  "optimum"  portfolio. 

Table  7  lists  references  for  reviews  of  different  types 
of  portfolio  selection  models.  Typically  portfolio  models 
are  mathematical  programming  models ,  such  as  goal 
programming.  The  characteristic  that  separates  these  models 
from  other  selection  models  is  the  existence  of  definite 
budget  and  other  resource  constraints  that  help  the  decision 
maker  analyze  how  an  increase/decrease  in  the  resources 
allocated  to  one  project  will  affect  other  projects.  The 
first  three  steps  of  the  selection  system — the  establishment 
of  goal  statements  and  a  classification  scheme  and  the 
choosing/ ranking  of  projects  within  each  class--have  been 
designed  to  provide  the  necessary  input  data  for  formulating 
such  a  model.  Therefore,  greatly  simplifying  the  modeling 
process. 

A  few  applications  of  this  combination  of  a  portfolio 
model  and  a  group  decision  making  technique  have  been  tested 
and  the  results  have  been  encouraging  [S-12] [C-5] [B-8 ] .   The 
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TABLE  7: 


Reviews  of  R&D  Portfolio  Selection  Models. 


Ref.   Author (s) 


Date  Description 


B-8   Booker  & 
Bryson 

C-l   Cetron, 

Martino,  & 
Roepcke 

G-l   Gear 


G-2   Gear,  Lockett 
&  Pearson 

J-2   Jackson 


M-l   Madey  & 
Dean 


S-4   Souder 


S-ll   Souder 


1985   A  recent  and  complete  review  of 
most  portfolio  models. 

1967  Well-quoted,  descriptive  review 
of  30  models. 


1974  A  good  discussion  of  practical 
&  theoretical  weaknesses  of 
several  portfolio  models. 

1971  A  thorough  discussion  of  9 
specific  portfolio  models. 

1983   Discussion  of  adv./disadv.  of 
major  classes  of  portfolio 
models. 

1985   Proposes  and  solves  a  mathe- 
matical programming  model  for 
an  aerospace  firm. 

1973   4  mathematical  programming  models 
are  compared  using  data  from  30 
actual  R&D  projects. 

1973   3  expected  value  models  are 

compared  for  analytical  utility 
and  managerial  acceptability. 
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basis  of  the  Souder  model  described  earlier  was  the  inte- 
gration of  a  Q-Sorting/Nominal-Interacting  process  with 
various  selection  models  including  portfolio  selection 
models  [S-10].  Another  well-quoted  article  by  Souder 
solicited  the  necessary  input  information  for  a  dynamic 
programming  algorithm  through  polling  and  interviews  [S-9]. 

A  relatively  novel  combination  of  a  group  decision 
making  technique  and  a  multiple  objective  goal  programming 
model  was  proposed  by  Khorramshahgol,  et  al.  [K-2]  in  1986. 
The  Delphi  technique  was  used  to  provide  the  objectives,  the 
priorities  among  objectives,  the  desired  level  of  attainment 
for  each  objective  (thus  transforming  it  into  a  goal),  and 
the  penalty  weights  for  the  under-  or  over-attainment  of 
each  goal.  This  information  was  used  to  formulate  a  linear 
goal  programming  model  which  was  solved  for  the  optimum 
portfolio. 

The  use  of  a  mathematical  programming  portfolio 
selection  model  as  the  sole  and  final  selection  instrument 
has  been  shown  to  be  an  unacceptable  option  for  most 
managers.  Most  managers  do  not  understand  the  complex 
mathematics  behind  the  model  and  consequently  will  not 
blindly  accept  the  solutions.  Therefore,  by  using  the  model 
to  stimulate  the  decision  making  process  of  the  committee, 
the  model  is  able  to  provide  valuable  analytical  support  in 
a  manner  that  is  acceptable  to  most  managers. 

The   result  of  this  final  phase  is  the  organization's 
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research  and  development  portfolio.  By  soliciting  the  input 
of  the  key  individuals  and  groups  within  the  organization, 
all  the  relevant  evaluative  information  has  been  utilized 
and  digested  in  the  selection  process.  Additionally,  and 
possibly  more  importantly,  the  decision  process  has  served 
to  provide  long-term  benefits  to  the  organization  in  terms 
of  improved  communication,  cooperation,  and  motivation. 

An  Example  Application 

The  proposed  R&D  project  selection  system  will  now  be 
applied  to  a  hypothetical  example  of  selecting  research  and 
development  projects  for  the  Strategic  Defense  Initiative, 
better  known  as  Star  Wars.  This  example  will  demonstrate 
how  the  system  must  be  adapted  to  the  specific  organization 
and  to  the  unique  set  of  R&D  projects  under  consideration. 

Referring  back  to  Figure  4,  the  four  phases  of  the 
selection  system  are  as  follows:  I)  establish 
organizational  goals,  II)  classify  projects.  III)  choose  or 
rank  projects  within  each  class,  and  IV)  pool  the  projects 
and  select  a  portfolio.  Phases  I  and  IV  are  implemented  by 
the  selection  committee  and,  therefore,  use  a  group  decision 
making  technique  while  Phases  II  and  III  are  implemented  by 
the  individual  R&D  units  or  managers . 

The  first  step  is  to  choose  a  group  decision  making 
technique   and  to  establish  the  goals  of   the   organization. 
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If  all  of  the  members  of  the  selection  committee  can  be 
brought  together  into  one  room,  a  Nominal-Interacting  group 
technique  could  be  used.  Otherwise,  if  the  members  are 
geographically  disperse,  a  Delphi  Survey  would  be  more 
appropriate.  The  committee  would  work  towards  statements  of 
the  goals  of  the  organization  such  as  the  following: 


1.  Produce   politically  rewarding  results  by 
November,  1988,  in  order  to  insure  future  funding. 

2.  Establish  the  feasibility  of  the  system  by 
1990. 

3.  Develop  a  working  system  by  2010. 

4.  Balance  project  completion  times  to  eliminate 
technical  bottle-necks. 


The  second  phase  involves  classifying  projects 
according  to  an  appropriate  classification  scheme. 
Therefore,  the  committee  must  first  decide  how  the  projects 
should  be  classified.  An  obvious  possibility  would  be  to 
classify  projects  according  to  their  expected  completion 
times.  From  the  goal  statements,  it  is  clear  that  the 
committee  is  integrally  concerned  with  completion  times. 
Projects  could  be  divided  which  have  completion  times  as 
follows: 
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<  2  yrs. 


3-5  yrs. 


6-10  yrs. 


11-17  yrs. 


>  18  yrs. 


*  satellite 

*  simulated  *  laser 

*  radar 

* 

satellite 

decoys 

model       power 
source 

systems 

shields 

C3I 

*  new 

*  satellite 

*  laser 

* 

plastics 

design    *  new 

metal 
alloys 

control 
system 

systems 

The  third  phase  of  the  selection  system  is  to  choose  or 
rank  projects  within  each  class.  Since  the  projects  have 
been  classified  according  to  expected  completion  time,  there 
are  certain  differences  between  the  classes  of  projects  that 
must  be  considered  in  choosing  an  appropriate  project 
selection  model. 

For  those  projects  which  are  expected  to  be  completed 
in  the  next,  say,  5  years,  more  data  will  be  available  with 
much  less  uncertainty.  Better  estimates  of  factors  such  as 
costs,  required  resources,  probabilities  of  success,  and 
actual  completion  times  will  be  available  for  these 
projects.  Therefore,  a  more  in-depth  analysis,  such  as  risk 
analysis,  is  possible  and  should  be  used  for  the  selection 
of  projects  within  these  classes.  As  the  projects' 
completion  times  extend  further  into  the  future,  more  of  the 
data  available  on  the  projects  is  of  a  qualitative  rather 
than  quantitative  nature.  Much  of  the  available  evaluative 
information  will   be  in  the  form  of  subjective   ratings   on 
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important  criteria.  Therefore,  models  that  can  use  this 
type  of  data,  such  as  scoring  models,  should  be  applied  for 
the  intermediate-range  projects.  For  those  projects  with 
expected  completion  time  greater  than  18  years,  a  simple 
checklist  comparison  may  be  most  appropriate. 

The  fourth  and  final  phase  is  to  pool  the  projects  and 
select  a  portfolio.  The  selection  committee  is  regrouped 
and  the  same  group  decision  making  technique  is  applied  as 
was  used  in  Phase  I .  A  model  such  as  Linear  Goal 
Programming  could  be  formulated  using  mathematical 
representations  of  the  four  goals  described  in  Phase  I . 
The  attainment  of  these  goals  could  be  optimized  subject  to 
alternative  budget  constraints  and  alternative  funding 
levels.  The  model  would  allow  the  decision  makers  to 
analyze  the  impact  of  not  only  changes  in  which  projects  are 
included  in  the  portfolio,  but  in  changes  in  the  funding 
levels  in  future  budgeting  periods.  The  sensitivity 
analysis  would  continue  until  the  committee  could  agree  on  a 
set  of  projects. 

This  example  demonstrates  how  the  proposed  system  can 
be  adapted  to  any  organization  by  integrating  the  specific 
goals  of  the  organization  into  the  total  selection  process. 
Additionally,  the  selection  of  an  appropriate  classification 
scheme  according  to  the  organizational  goals  insures  that 
the  project  selection  process  is  implemented  in  a  manner 
consistent  with  those  goals. 
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From  the  example  above,  it  is  evident  that  in  order  to 
effectively  utilize  this  project  selection  system,  a 
reasonable  classification  scheme  must  be  determined  for  the 
specific  organization  and  the  characteristics  of  the  mix  of 
projects  under  evaluation.  Although  there  are  an  infinite 
number  of  possible  classification  schemes,  a  few  schemes 
have  dominated  the  literature.  These  schemes  will  be 
discussed  in  detail  in  the  following  chapter. 
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CHAPTER  FIVE: 


CLASSIFICATION  SCHEMES 


The  diversity  of  the  research  and  development  projects 
being  considered  by  any  organization  at  any  one  point  in 
time  causes  decision  makers  to  directly  compare  projects 
that  have  only  a  few  common  characteristics.  The  mix  of 
projects  depends  not  only  upon  the  organization,  but  each 
organization  will  face  a  completely  different  mix  every  time 
a  project  selection  decision  is  made.  This  diversity  of 
alternative  projects  greatly  complicates  the  selection 
process  to  a  point  of  unmanageability. 

This  problem  can  be  simplified  if  some  of  the  major 
differences  between  projects  are  eliminated  before 
evaluative  comparisons  are  made.  This  is  accomplished  by 
grouping  the  projects  into  like  groups,  i.e.  classifying  the 
projects.  All  the  projects  in  each  class  will  share  enough 
characteristics  that  they  can  be  realistically  compared. 
The  selection  process  is,  therefore,  simplified  to  allow  for 
the  more  practical  application  of  R&D  selection  methods. 

The  second  advantage  of  classifying  projects  is  helping 
to  insure  a  balanced  R&D  portfolio  for  the  organization.  By 
classifying  projects  according  to  the  strategic  goals  of  the 
organization,  it  can  be  assured  that  each  strategic  class  of 
projects   is  represented  in  the  organization's   final   port- 
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folio  of  R&D  projects.   The  end  effect  is  a  more  "balanced" 
R&D  portfolio. 

Since  each  organization  will  classify  projects 
according  to  its  own  set  of  strategic  goals,  the  number  of 
classification  schemes  is  just  as  great  as  the  number  of 
unique  sets  of  organizational  goals.  The  Industrial 
Research  Institute  made  a  similar  conclusion  in  a  study  on 
basic  research  conducted  for  the  National  Science 
Foundation  [N-2]: 


The  manner  in  which  each  firm  categorized  its 
R&D  activity  was  typically  tailored  to  their  own 
organizational  structure  and  function.  In 
addition,  in  only  one  industry--aerospace--did  the 
investigators  find  any  two  firms  with  the  same 
generic  categories.  However,  given  this  diversity 
of  descriptors,  could  any  set  of  categories 
achieve  industrial  acceptance?  Probably  not. 


Therefore,  no  one  set  of  categories  will  be  proposed  or 
recommended  for  use  by  all  organizations.  Each  organization 
must,  after  establishing  its  own  unique  statements  of  its 
goals,  decide  on  an  appropriate  classification  scheme. 

The  following  sections  will  discuss  several  common 
classification  schemes  for  research  and  development 
projects.  Almost  all  discussions  in  literature  in  this  area 
that  refer  to  the  classification  of  projects  use  one  of  the 
following  schemes. 

The  Industrial  Research  Institute,  in  an  attempt  to 
devise   a  universally  acceptable   classification  scheme, 
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considered  classifying  R&D  projects  by  how  the  work  is  done 
(fundamental,  basic,  applied),  by  where  it  is  done  (central 
labs,  divisional  outposts,  semi-works,  bench),  by  whether 
the  research  is  product  or  process  oriented,  or  by  why  the 
work  is  done  (exploratory,  high  risk  business  development, 
support  of  existing  business)  [G-3].  The  how,  why,  and 
product/process  schemes  are  discussed  in  the  following 
sections.  The  question  of  where  the  work  is  done  will  be 
taken  up  in  the  following  chapter  on  organizational  factors. 

Life  Cycle  Stage 

By  definition,  the  term  "research  and  development" 
suggests  a  differentiation  between  projects  according  to 
their  life  cycle  stage--from  research  to  development.  Many 
obvious,  and  some  not  so  obvious,  changes  take  place  in  the 
evaluative  information  and  criteria  as  a  project  progresses 
from  an  idea  to  an  actual  product  or  process.  Liberatore 
and  Titus  [L-4]  suggest  some  of  these  changes  that  take 
place  in  the  evaluation  of  R&D  projects  as  they  progress 
from  "R"  to  "D" : 


Therefore,  the  organizational  context  in 
which  R&D  resource  allocation  occurs  must  be  con- 
sidered by  management  scientists  in  the  develop- 
ment of  appropriate  methods.  For  example,  the 
availability  and  reliability  of  data,  the  level  of 
detail  for  the  statement  of  project  goals,  the 
time  horizon  for  project  activity,  the  criteria 
for  successful  project  completion  and  the  struc- 
ture of  the  R&D  and  supporting  groups,  among  other 
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factors,   are  quite  different  in  "R"  and  "D"  envi- 
ronments. 


Divisions  based  on  the  different  stages  of  the 
project's  life  are  the  most  commonly  used  in  literature. 
This  transition  from  "R"  to  "D" ,  though,  is  not  a  discrete 
process  with  well-accepted  and  divisible  stages.  Rather, 
the  process  is  continuous  with  each  author  or  organization 
having  a  different  opinion  on  the  number  and  definition  of 
the  intervening  stages. 

For  example,  Baker  [B-5]  summarizes  a  project's  life 
into  the  following  stages:  research,  development, 
engineering,  and  manufacturing.  Ansoff  [A-l],  in  an 
analysis  of  applied  projects,  suggests  three  divisions  of 
R&D  projects:  basic,  applied,  and  product  development. 
Finally,  Balderston  [B-6]  proposes  more  detailed  classes: 
basic  research,  applied  research/exploratory  development, 
advanced  development,  engineering  development,  production 
engineering,  and  product  or  service  projects.  Each  of  these 
divisions  differ  with  respect  to  the  level  of  the  following 
attributes:  internal  scope,  external  relatedness,  need  for 
integration,  state  of  the  art,  team  experience,  and  the  need 
for  flexibility. 

Several  papers  have  concentrated  on  describing  the 
differences  between  projects  in  various  life  cycle  stages. 
Nason  [N-l]  in  his  paper,  "Distinctions  between  Basic  and 
Applied   in  Industrial  Research,"  summarizes  the  results   of 
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three  studies  in  this  area  [N-2] [F-2] [P-l] .  More  recently, 
Leifer  and  Triscari  [L-l]  compared  the  aspects  of  managing 
research  versus  development  projects. 

Easily  the  most  notable  use  of  this  classification 
scheme  is  by  the  National  Science  Foundation  (NSF).  The  NSF 
proposes  dividing  projects  among  basic  research,  applied 
research,  and  development.  These  categories  are  defined  as 
follows  [N-2]: 


Basic  Research:  Original  investigations  for  the 
advancement  of  scientific  knowledge  not  having 
specific  commercial  objectives,  although  such 
investigations  may  be  in  fields  of  present  or 
potential  interest  to  the  reporting  company. 

Applied  Research:  Investigation  directed  to  the 
discovery  of  new  scientific  knowledge  having  spe- 
cific commercial  objectives  with  respect  to  pro- 
ducts or  processes.  This  definition  differs  from 
that  of  basic  research  chiefly  in  terms  of  the 
objectives  of  the  reporting  company. 

Development:  Investigations  which  represent  tech- 
nical activity  concerned  with  non-routine  problems 
which  are  encountered  in  translating  research 
findings  or  other  general  scientific  knowledge 
into  products  or  processes. 


These  categories  and  definitions  are  used  by  companies  in 
reporting  research  and  development  expenditures.  The 
Industrial  Research  Institute  (IRI)  confirms  that  most 
organizations  use  these  categories  for  external  reporting 
but  use  their  own  categories  for  internal  reporting  [N-2]. 

The  same  1978  study  reports  the  percent  of  expenditures 
allocated  to  each  of  these  categories  as  follows: 
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I.R.I. 

N.S.F. 

Study 

Estimate 

Basic  Research 

5.3% 

3.7% 

Applied  Research 

28.6% 

22.0% 

Development 

66.1% 

74.3% 

Expenditures  for  research  and  development  in  these  three 
areas  is  also  reported  by  industry.  Clearly,  expenditures 
for  development  projects  dominate  the  total  R&D  expenditures 
while  basic  research  projects  make  up  only  a  small  fraction 
of  the  typical  R&D  portfolio. 

Thus  far,  only  the  changing  nature  of  R&D  projects  as 
they  progress  from  basic  research  to  commercial  production 
has  been  proposed  as  a  justification  for  dividing  R&D 
projects  by  their  life  cycle  stages.  Ramsey  [R-l]  suggests 
an  additional  aspect  of  a  projects  life  cycle  that  supports 
this  classification  scheme.  Ramsey's  new  product  process 
stage  model  is  presented  in  Figure  5.  In  addition  to  the 
division  of  the  life  cycle  into  6  stages,  Ramsey  proposes  4 
decisions  related  to  the  evaluation  of  projects  as  they 
progress  from  idea  generation  to  market  introduction. 

Decision  I  is  concerned  with  the  screening  of  ideas  to 
pick  the  ones  for  which  to  perform  a  business  analysis.  The 
second  decision  (II)  involves  choosing  projects  for 
technical  development.  Decision  III  reviews  the  projects 
from  decision  II  to  determine  if  the  project  should  be 
continued.   Finally,  the  purpose  of  decision  IV  is  to  review 
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FIGURE  5:   Ramsey's  Decision  Stages.  [R-l] 
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the  projects  before  commercialization  to  determine  if  any 
projects  should  be  terminated  to  minimize  losses. 

The  division  of  a  project  into  the  stages  of  its  life 
cycle  can  help  an  organization  in  the  analysis  of  the 
feasibility  of  continuing  work  on  the  project.  It  is  not 
necessary  to  use  the  exact  divisions  proposed  by  Ramsey  in 
order  to  identify  the  key  decision  stages  of  an  R&D  project. 
These  decision  stages  could  be  identified  on  any  alternative 
division  of  the  stages  of  the  project's  life  cycle  depending 
on  the  evaluation  process  of  the  particular  organization. 

Most  authors,  when  they  introduce  or  review  R&D  project 
selection  models,  suggest  the  type  of  projects  for  which  the 
models  are  most  suited.  This  concern  over  the  applicability 
of  any  certain  model  is  justified  by  the  many  unique 
characteristics  of  the  projects  at  different  stages  of 
development.  A  summary  of  some  of  these  changing 
characteristics  would  include  changes  in  the  amount  and 
certainty  of  evaluative  information,  the  goals  of  the 
project,  the  time  horizon  for  completion,  and  the  structure 
of  the  R&D  group. 

The  Industrial  Research  Institute  [N-2]  stressed  the 
importance  of  separating  basic  research  projects  from  other 
projects  for  selection  purposes.  Their  study  showed  that 
return  on  investment  (ROD  methods  are  increasingly  being 
used  in  industry.  "The  extent  of  'quantification,'  in  an 
ROI   rather  than  mathematical  programming  sense,   in  private 
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sector  R&D  management  has  essentially  placed  basic  research 
efforts  in  direct  competition  with  development  efforts--but 
this  game  is  played  with  rules  which,  by  definition,  favor 
development. " 

Both  Souder  [S-8][S-10]  and  Krawiec  [K-3]  suggest 
different  models  for  use  in  evaluating  projects  during  the 
stages  of  the  project's  life.  Cetron  et  al.  [C-l]  reviewed 
30  models  in  1967  and  rated  each  on  its  applicability  to 
projects  in  the  research,  exploratory  development,  advanced 
development,  and  engineering  development  stages.  Baker  and 
Pound  [B-3]  evaluated  the  use  of  three  representative  models 
(scoring,  expected  value,  and  dynamic  programming)  for  the 
selection  of  basic  research,  applied  research,  and  develop- 
ment projects.  Moore  and  Baker  [M-6]  recommend  a  scoring 
model  for  research  and  exploratory  development,  economic  or 
constrained  optimization  for  advanced  and  engineering  devel- 
opment, and  risk  analysis  for  new  product  decisions. 

Although  the  classification  of  R&D  projects  by  the 
stages  of  the  project  life  cycle  is  most  often  used  in 
literature,  there  has  more  recently  been  a  trend  toward 
classifying  projects  by  their  business  objective.  The 
Industrial  Research  Institute  has  been  instrumental  in 
changing  the  thinking  of  many  researchers  toward  this  class- 
ification scheme.  The  following  section  will  discuss  this 
so-called  "business  classification." 
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New  vs.  Existing  Business 

The  approach  of  any  decision  maker  toward  any  decision 
is  based  upon  his/her  past  experience  and  knowledge  in  the 
area  of  the  question  at  hand.  Clearly,  if  the  decision 
maker  has  no  past  experience  with  a  certain  project,  a 
higher  level  of  risk  is  involved  in  the  evaluation  of  that 
project.  On  the  other  hand,  if  the  project  is  concerned 
with  a  simple  extension  of  existing  operations,  the  decision 
maker  will  be  in  a  better  position  to  estimate  the  outcome 
of  the  project.  For  these  reasons,  projects  are  often 
classified  by  the  "newness"  of  the  project  to  the  organ- 
ization. 

This  classification  scheme  is  often  referred  to  as 
classification  by  the  objective  of  the  project.  This  scheme 
is  largely  based  on  the  different  levels  of  information 
available  to  the  decision  maker.  Without  the  experience  and 
historical  data  on  which  to  base  forecasts,  estimates  of  the 
probability  of  technical  and  commercial  success,  time  to 
completion,  and  expected  costs  and  revenues  will  contain 
much  more  uncertainty.  All  of  these  estimates  are  important 
inputs  into  any  project  selection  model. 

The  Industrial  Research  Institute  (IRI)  is  a  group  of 
approximately  250  industrial  firms  designed  to  collect  data 
and  perform  research  on  the  industrial  research  activities 
of   its  members.     The  group  has   struggled  with   this 
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classification  scheme  throughout  the  1970s  and  1980s.  In 
1971,  IRI  rejected  all  other  classification  schemes  in  favor 
of  classifying  R&D  by  the  objective  of  the  work  (why  it  is 
done)  [G-3].  The  three  divisions  identified  by  IRI  were  as 
follows: 

Exploratory  Research: 

For  advancing  knowledge  of  phenomena  of 
general  company  interest 

For  finding  major  new  high  risk  business 
developments 

High  Risk  Business  Development: 

Specific  major  programs  aimed  at  new 
businesses  or  other  developments  of  potentially 
high  impact,  and  which  involve  higher-than-normal 
risk 

Support  of  Existing  Business: 

For  maintaining  or  improving  the  profit- 
ability of  established  business 

Two  of  the  common  characteristics  of  exploratory 
research  projects  are:  1)  the  responsibility  for  the 
selection  of  projects  lies  within  the  R&D  department;  and 
2)  the  selection  process  tends  to  be  simple,  unsophisti- 
cated, and  qualitative  in  nature. 

Projects  classified  as  supporting  existing  business 
usually  involve  cost-reduction  or  manufacturing  process 
efforts  or  the  development  of  products  to  round  out  the 
current  product  line.  The  selection  of  such  projects  is 
characterized  by  the  availability  of  much  quantitative   data 
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that  is  used  in  simple  economic  projections. 

The  level  of  decision  making  and  the  risk  involved  are 
two  of  the  prominent  characteristics  of  high  risk  business 
development  projects.  Since  these  projects  imply  a  new 
direction  for  the  organization,  it  is  not  uncommon  for  such 
decisions  to  be  made  at  very  high  levels  in  the  hierarchy  of 
the  organization.  Although  a  few  more  sophisticated 
selection  methods  are  used  here,  standard  economic  projec- 
tions are  still  the  norm.  Clearly,  the  risks  associated 
with  these  projects  is  much  higher  than  for  the  previous  two 
categories. 

In  1978,  IRI  modified  their  classification  scheme  to 
include  both  the  business  objective  and  the  life  cycle  stage 
of  each  project.  This  matrix  classification  scheme  will  be 
discussed  in  the  "Combinations"  section  below.  IRI  also 
divided  the  "support  of  existing  business"  category  into 
"maintenance  or  modifications"  and  "major  products  and/or 
new  techniques"  and  redefined  the  category  "high  risk  busi- 
ness development"  as  "diversification"  [1-1]. 

Currently,  the  Institute  uses  similar  classifications 
in  an  annual  survey  of  its  members  on  the  changes  in  the 
funding  levels  for  3  categories  of  R&D  (as  well  as  other 
statistics  on  R&D).  The  classifications  currently  used  are 
as  follows: 

1)  Support  of  Existing  Business 
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2)  Directed  Basic  Research 

3)  New  Business  Projects 

The  1987  survey  reported  that  a  number  of  members  planned  to 
increase  their  support  for  projects  dealing  with  new 
business  projects.  The  other  two  categories  of  R&D  were 
less  assured  of  receiving  more  support  [M-7]. 

The  lack  of  information  available  to  evaluate  new 
business  projects  increases  the  risk  to  the  organization  of 
adopting  these  projects.  This  factor  accounts  for  most  of 
the  differences  cited  among  projects  with  different  business 
objectives.  Gee  [G-3]  reports  the  conclusions  made  by  the 
1971  IRI  committee  on  the  differences  between  projects  with 
different  business  objectives  as  do  several  other  authors 
[C-3][B-10].  Keefer  [K-l]  proposes  a  similar  classification 
scheme  and  outlines  a  rating  scale  for  various  criteria 
associated  with  each  class  of  projects. 

Although,  like  the  stages  of  the  project  life  cycle, 
the  business  objectives  of  various  projects  can  be  described 
and  divided  an  endless  number  of  ways,  the  business  class- 
ification schemes  usually  share  two  common  characteristics. 
First,  a  category  for  basic,  fundamental,  or  exploratory 
research  is  separated  from  the  other  categories.  Secondly, 
a  clear  distinction  is  made  between  projects  that  are 
concerned  with  improving  or  expanding  existing  products  or 
processes   versus  those  projects  concerned  with  diversifying 
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into  new  business  lines. 

Clearly,  this  classification  scheme  would  not  be 
appropriate  for  certain  types  of  organizations  such  as 
government  agencies,  non-profit  organizations,  and 
educational  institutions. 

After  life  cycle  stage  and  business  objective,  the  next 
most  popular  classification  scheme  in  literature  is  the  one 
that  separates  projects  that  involve  the  development  of  new 
products  from  those  concerned  with  new  processes.  As  will 
be  discussed  in  the  next  section,  these  differences  between 
these  two  types  of  projects  can  often  be  limited  to  the 
extent  to  which  outside  factors  effect  the  success  of  the 
project. 

Product  vs.  Process 

The  distinction  between  product  and  process  R&D  efforts 
seem  to  be  one  of  the  first  classification  schemes  to  be 
proposed.  More  recently,  this  distinction  is  often  made 
only  as  a  sub-classification  for  another  classification 
scheme.  A  few  of  these  combination  schemes  will  be 
discussed  later  in  this  chapter. 

Often  the  division  between  projects  on  the  basis  of 
whether  they  involve  products  or  processes  is  not  inter- 
preted literally.  Sometimes  what  is  implied  by  this 
distinction   is   similar  to  the  business   objective   scheme 
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proposed  above.  Under  this  interpretation,  "product- 
oriented"  projects  are  those  involving  marketable  products 
to  be  offered  to  the  customers  of  the  organization. 
"Process-oriented"  projects  are  those  concerned  with  the 
manufacturing  or  other  organizational  processes,  i.e.  cost- 
reduction  efforts. 

The  main  difference  between  projects  under  these 
definitions  is  the  extent  to  which  factors  outside  the 
organization  (competition,  demand  for  products,  etc.)  affect 
the  project.  For  example,  cost-reduction  efforts  can  be 
undertaken  without  any  analysis  of  the  business  environment 
whereas  such  an  analysis  is  vital  to  the  success  of  the  so- 
called  "product-oriented"  project. 

A  1978  survey  by  IRI  on  successful  innovations  [F-2] 
provided  some  insights  into  the  appropriateness  of  this 
classification  scheme.  The  survey  respondents  were  asked  to 
classify  their  "successful  innovation"  as  concerned  with  a 
product,  a  process,  or  both.  Fifty-nine  percent  of  the 
projects  were  classified  as  dealing  with  products  and  19% 
were  processes.  The  interesting  result  was  the  response  of 
22%  of  those  surveyed  who  reported  that  their  projects  dealt 
with  both  products  and  processes.  Additionally,  one  inno- 
vation was  labeled  as  a  "service". 

These  results  cast  some  doubt  on  the  practicality  of 
this  classification  scheme.  First  of  all,  since  22%  of  the 
projects  were  both  products  and  processes,  these  classes  are 
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not  mutually  exclusive.  Similar  to  both  of  the  schemes 
discussed  above,  the  divisions  between  the  classes  are  again 
continuous,  not  discrete.  Secondly,  the  possibility  that 
R&D  projects  are  solely  concerned  with  the  development  of 
new  services  introduces  another  weakness  of  this  scheme. 
Perhaps  a  third  category  for  services  should  be  included  to 
account  for  this  increasingly  important  area  of  research  in 
our  economy. 

The  advantage  to  this  classification  scheme  is  the 
separation  of  "process"  projects  for  which  only  factors 
internal  to  the  organization  must  be  included  in  the 
selection  model.  The  evaluation  and  selection  of  these 
projects  would  be  greatly  simplified.  Whaley  and  Williams 
[W-l]  discuss  the  basic  differences  between  product  and 
process-oriented  projects.  Disman  [D-4]  proposes  selecting 
projects  according  to  a  return  on  investment  (ROD  model; 
a  different  model  for  products  and  processes.  The  IRI 
Research  on  Research  Committee  has  recommended  possible 
criteria  for  research,  product  development,  and  process 
development.  These  recommendations  are  reported  by  Becker 
[B-7]. 

As  discussed  earlier,  there  are  several  key  weaknesses 
in  the  product/process  classification  scheme.  In  fact,  any 
attempt  to  classify  the  diversity  of  R&D  projects  into  a 
finite  number  of  categories  will  never  be  complete.  All 
three   of  the  proposed  classification  schemes  discussed  thus 
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far  (those  most  cited  in  literature)  tend  to  be  both 
subjectively  defined  and  often  incomplete  in  coverage.  To 
overcome  this  problem,  some  authors  have  proposed  class- 
ification schemes  based  on  a  combination  of  two  of  the  above 
schemes.  Several  of  these  schemes  will  be  presented, 
including  a  well-documented  matrix  by  the  Industrial 
Research  Institute. 

Combinations 

Certain  authors  have  proposed  using  not  one  of  the 
above  classification  schemes,  but  combining  two  of  them  in 
order  to  more  narrowly  define  the  categories  of  research  and 
development.  These  more  detailed  classification  schemes  may 
be  necessary  for  a  detailed  analysis  of  the  overall  R&D 
effort  on  an  organizational  scale,  or  when  the  number  of 
alternative  projects  is  so  large  that  3  to  4  categories  will 
not  divide  the  pool  of  projects  into  groups  of  manageable 
size.  The  IRI  has  proposed  such  a  combination  in  a  matrix 
form  that  combines  the  stages  of  R&D  with  the  business 
objective  of  the  projects. 

Liberatore  [L-3]  proposed  a  unique  approach  to 
classifying  projects  in  his  paper  on  the  analytical 
hierarchy  process  (AHP).  The  typical  classification  of 
product  vs .  process  vs .  exploratory  are  modeled  as  one  level 
of    a  hierarchy  while   another   level  of   the   hierarchy 
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contained  the  business  objectives:  maintenance  vs.  expan- 
sion vs.  diversification.  All  possible  combinations  of  the 
two  sets  of  categories  are  diagrammed. 

Becker  [B-7]  reports  the  classification  scheme  devised 
by  the  IRI  Research  on  Research  Committee  in  1980  which 
combines  a  business  objective  classification  divided  among 
product  and  process  efforts  as  follows: 


Research 

Product  Development—Maintaining  Business 

Process  Development—Maintaining  Business 

Major  Product  Modification — Business  Expansion 

Major  Process  Modif ication--Business  Expansion 

Product  Development—New  Business 

Process  Development— New  Business 


Except  for  the  lone  category  of  "Research" ,  this  scheme 
could  easily  be  modeled  as  a  2  X  3  matrix  with  Product  and 
Process  efforts  on  one  axis  and  the  three  business 
objectives  (Maintaining  Business,  Business  Expansion,  and 
New  Business)  on  the  other  axis. 

The  IRI  in  1978  published  a  report  entitled 
"Definitions  of  Research  and  Development"  [1-1].  This 
report  outlined  another  matrix  combining  the  business 
objectives  with  what  it  calls  the  "R&D  Dimensions."  See 
Figure  6.  The  latter  set  of  categories  included  research, 
development,  and  technical  service.  The  Institute  defined 
these  terms  as  follows: 


Research  is  planned  search  or   critical   invest- 
igation aimed  at  discovery  of  new  knowledge  with 
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the  objective  that  such  knowledge  will  be  useful 
in  developing  new  products/processes/services,  or 
in  bringing  about  a  significant  improvement  to 
existing  products/processes/services . 


Development  is  the  translation  of  research 
findings  or  other  knowledge  into  a  plan  or  design 
for  new,  modified,  or  improved  products/processes/ 
services  whether  intended  for  sale  or  use.  It 
includes  the  conceptual  formulation,  design,  and 
testing  of  product/process/service  alternatives; 
the  construction  of  prototypes;  and  the  operation 
of  initial,  scaled-down  systems  or  pilot  plants. 
It  does  not  include  routine  or  periodic  alter- 
ations to  existing  products,  production  lines, 
manufacturing  processes,  service,  and  other  on- 
going operations  even  though  those  alterations  may 
represent  improvements. 


Technical  Service  is  work  performed  by  R&D 
personnel  (usually  requested  by  other  areas  of  the 
company,  such  as  manufacturing,  marketing,  or 
corporate  management).  It  involves  the  appli- 
cation of  existing  technology  to  company  problems 
in  the  existing  business. 


The  business  dimensions.  Existing  Business  and 
Diversification,  are  listed  on  the  top  of  the  figure.  The 
Existing  Business  is  further  divided  into  Maintenance  or 
Modifications  and  Major  Products  and/or  New  Techniques. 

The  matrix  has  numerous  features  which  are  designed  to 
help  organizations  in  defining  their  R&D  projects.  Notice 
that  the  technical  service  category  falls  only  under  the 
Maintenance  or  Modification  column.  According  to  the 
definition  of  technical  service,  this  work  involves  only 
improvements  in  the  existing  operations  of  the  business. 
The   dotted  line  separating  the  business  dimensions   in  the 
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research  row  indicates  that  some  research  will  not  have  a 
defined  commercial  purpose  at  the  outset.  Therefore,  the 
division  of  research  efforts  between  existing  and  new 
business  may  not  be  discernible.  Additionally,  the  research 
and  development  rows  contain  slashes  to  allow  the  addition 
of  sub-classifications  as  desired.  Finally,  the  heavy  boxes 
represent  totals  that  IRI  suggest  should  be  used  by  all 
companies  as  a  tool  for  analyzing  R&D  expenditures  and 
budgets . 

This  matrix  is  possibly  the  most  complete  classifi- 
cation scheme  proposed  in  literature.  By  combining  the  two 
most  used  classification  schemes  and  clearly  defining  each 
of  the  categories  used,  the  scheme  could  promote  some  level 
of  uniformity  among  IRI -member  organizations.  Yet,  with  the 
allowance  for  each  individual  firm  to  use  its  own  sub- 
classifications  and  totals  in  the  matrix,  the  scheme  becomes 
a  tool  for  a  firm  in  defining  the  classification  scheme  that 
best  fits  its  own  particular  needs. 

Recommendations 

From  the  multitude  of  possible  R&D  project  classifi- 
cation schemes,  the  three  most  used  schemes,  as  well  as  some 
possible  combinations  of  the  three,  have  been  presented  and 
discussed  in  this  chapter.  None  of  these  schemes  are  going 
to  earn  the  acceptance  of  all  R&D-performing  organizations 
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or  even  a  majority  of  them.  Each  has  its  own  advantages  and 
disadvantages.  Each  may  be  best  suited  for  applications 
other  than  project  selection.  These  schemes  should  be  used 
as  a  base  point  from  which  organizations  can  decide  upon  an 
appropriate  classification  scheme  for  their  own  organiza- 
tion and  set  of  R&D  alternatives. 

Although  the  research  and  development  process  is 
continuous,  it  is  common  among  practitioners  to  break  up 
this  process  into  a  finite  number  of  stages.  Even  the 
division  between  research  and  development  is  such  a  class- 
ification. The  many  changing  aspects  of  projects  as  they 
progress  through  these  stages--however  they  are  defined-- 
make  the  classification  of  projects  by  stage  a  natural  one. 
The  changes  in  projects  as  they  develop  include  the 
availability  and  certainty  of  input  data,  the  goals  for 
successful  completion  of  the  project,  the  time  horizon  for 
completion,  and  the  structure  of  the  R&D  and  supporting 
groups. 

The  classification  of  projects  according  to  their 
business  objectives  (new  vs.  existing  business)  is  based 
upon  differences  in  the  experience  of  the  decision  makers 
between  projects  concerned  with  new  versus  existing 
businesses.  The  degree  of  risk  associated  with  new  business 
efforts  must  also  be  considered  when  modeling  selection 
decisions.  This  scheme  will  have  no  relevance  to  many 
public  and  non-profit  organizations. 
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The  IRI  matrix  is  clearly  the  most  complete  and  best 
defined  classification  scheme  presented  in  literature.  It 
combines  the  two  most  popular  schemes  in  a  logical, 
practical  manner.  Yet,  IRI  recommends  that  at  least  seven 
categories  be  used  by  all  companies.  Unless  an  organization 
feels  a  need  for  this  level  of  detail,  the  increase  in  the 
number  of  categories  will  proportionately  increase  the 
amount  of  work  involved  in  project  selection.  This 
increased  workload  will  come  without  any  significant 
additional  benefits.  It  is  important  to  remember  that  the 
IRI  matrix  was  designed  for  reporting  and  analyzing  R&D 
expenditures,  not  for  project  selection  purposes.  A 
somewhat  simpler  scheme  probably  would  be  preferable  for  the 
purpose  of  project  selection. 

The  classification  schemes  presented  in  this  chapter 
should  be  seriously  considered  when  choosing  from  the 
endless  number  of  possible  schemes  for  several  reasons. 
First,  these  schemes  have  been  most  commonly  proposed  in  the 
literature  and  most  used  in  practice.  Secondly,  model 
makers  will  find  that  much  more  information  is  available  on 
the  applicability  of  selection  models  with  respect  to  these 
classification  schemes  than  any  others. 

It  cannot  be  over-emphasized  that  the  classification 
scheme  used  by  each  organization  must  be  designed  to  best 
aid  the  decision  makers  in  reaching  the  goals  for  their 
specific   organization.    When  choosing  a   "best"   classifi- 
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cation  scheme  for  a  particular  organization,  the  committee 
must  consider  the  set  of  alternative  projects,  the  structure 
of  the  R&D  unit(s),  and  how  the  classification  scheme  can 
help  the  organization  best  "balance"  its  R&D  effort  in  terms 
of  its  strategic  goals. 

Some  of  the  typical  differences  in  the  approaches  of 
the  various  organizations  that  perform  R&D  is  the  topic  of 
Chapter  Six.  The  type,  structure,  and  size  of  the  organi- 
zation will  all  impact  the  manner  in  which  the  overall 
proposed  selection  process  is  implemented. 
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CHAPTER  SIX: 
ORGANIZATIONAL  FACTORS 

The  research  and  development  project  selection  system 
proposed  in  Chapter  Four  is  often  defined  in  relatively 
indefinite  terms.  The  reason  for  this  design  is  to 
encourage  different  organizations  to  develop  their  own 
specific  system  to  meet  their  own  specific  needs.  An  overly 
detailed  system  would  only  be  helpful  for  a  few  specific 
organizations  that  happen  to  be  faced  with  a  selection 
decision  like  the  one  modeled.  This  chapter  is  designed  to 
introduce  a  number  of  organizational  factors  that  affect  the 
way  in  which  different  organizations  see  the  R&D  project 
selection  decision  and,  therefore,  how  they  should  apply  the 
proposed  R&D  project  selection  system. 

The  first  organizational  factor  is  the  type  of 
organization.  A  governmental  agency  and  a  small  research 
firm  will  approach  the  problem  from  completely  different 
standpoints  and,  therefore,  must  adapt  the  system  in 
different  ways  to  make  it  work  for  them.  Additionally,  the 
structure  and  size  of  the  organization  have  unique  effects 
on  the  application  of  the  proposed  R&D  project  selection 
system. 
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Type  of  Organization 

Although  much  of  the  literature  has  focused  on  the  R&D 
project  selection  decisions  facing  large  corporations  or 
federal  agencies,  there  are  a  number  of  different  types  of 
organizations  that  perform  research  and  development.  All  of 
these  types  will  face  R&D  project  selection  decisions  but 
within  dramatically  different  contexts. 

Table  8  lists  and  briefly  describes  seven  classes  of 
organizations  that  do  most  of  the  research  and  development 
in  the  United  States.  The  main  difference  between  these 
types  of  organizations  is  the  goal  structure  of  the  organi- 
zation. For  example,  a  significant  difference  is  found 
between  the  profit-maximizing  corporate  R&D  laboratories  and 
government  laboratories.  The  reliance  on  federal  R&D 
expenditures  is  a  significant  aspect  of  the  functioning  of 
many  of  these  organizations  such  as  government  laboratories, 
federal  contract  research  centers,  and  educational  insti- 
tutions. Contract/customer  relationships  also  play  an 
important  role  in  some  types  but  are  completely  unrelated  to 
others. 

The  political  considerations  inherent  in  any  R&D 
organization  dealing  with  the  federal  government  often  play 
a  significant  role  in  the  R&D  project  selection  process. 
The  availability  of  funds  in  certain  areas  of  research  often 
dominate   the   selection  process  for  an  organization   that 
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TABLE  8:   Seven  Types  of  R&D  Organizations.  [F-3] 


Government  Laboratories 


whose  budgets 
government. 


government  owned   facilities 
and  objectives  are  set  by  the   federal 


2)  Federal  Contract  Research  Centers:  have  a  principal 
(although  not  necessarily  exclusive)  and  continuing 
obligation  to  one  sponsoring  federal  agency. 

3)  Not-for-Prof it  Institutions:  publicly  or  privately 
owned;  occupy  a  position  midway  between  academic 
research  and  R&D  as  performed  in  a  profit-making 
organization. 

4)  Independent  Nonprofit  Organization:  truly  independent 
nonprofit  research  organizations  which  are  funded  by 
private  supporters  and  donations. 

5)  Corporate  R&D  Laboratories:  for  the  purposes  of 
supporting  manufacturing  and  marketing  operations;  are 
funded  largely  from  corporate  profits;  tend  to  be  more 
applied,  product-oriented. 

6)  Educational  Institutions:  tend  to  have  more  modest 
capital  equipment  investments  but  can  perform  certain 
kinds  of  fundamental  R&D  of  high-quality  and  (generally) 
at  less  cost. 

7)  Private  Profit-Making  Institutions:  purpose  of  making  a 
profit  on  contract  R&D;  have  advantage  over  educational 
institutions  in  that  they  devote  their  entire  resources 
to  R&D. 
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relies  on  government  funding.  Government  agencies  must 
also  consider  the  political  funding  process.  The  Office  of 
Technology  Assessment  published  a  report  in  1986  entitled, 
"Research  Funding  as  an  Investment:  Can  We  Measure  the 
Returns?"  [0-1].  This  study  provides  an  excellent  survey  of 
the  practice  of  R&D  project  selection  and  management  in  both 
the  public  and  private  sectors.  Applications  of  R&D  project 
selection  techniques  in  government  settings  are  also 
discussed  by  Brickman  [B-9]  and  Cook  and  Seiford  [C-6]. 

In  addition  to  the  concentration  of  literature  on 
industry  as  a  group,  certain  specific  industries  receive 
more  attention  than  others.  Typically,  this  concentration 
of  interest  is  based  on  higher-than-normal  expenditures  on 
research  and  development  for  that  industry.  Most  obviously, 
these  industries  stand  to  gain  more  by  making  better 
selection  decisions.  Additionally,  some  papers  are  written 
by  employees  or  consultants  in  these  industries  who  have 
tested  selection  methods  and  are  reporting  the  results  in 
literature. 

An  IRI  survey  in  1977  [N-0.3]  reports  the  expenditures 
of  about  40  industries  on  basic  and  applied  research,  and 
development.  More  recently,  IRI  reports  the  annual  R&D 
expenditures  as  a  percent  of  sales  for  selected  industries 
each  year  in  its  Annual  R&D  Trends  Survey  [M-7.5].  More 
specifically,  the  pharmaceutical  [F-l] [B-7.5] ,  petroleum  [M- 
5],   and  chemical   [W-4]  industries  have   received   special 
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attention  in  the  literature. 

As  noted  before,  these  types  of  organizations  differ 
mainly  in  their  goal  structures.  Therefore,  it  is  important 
that  these  differences  are  accounted  for  in  the  implemen- 
tation of  the  proposed  project  selection  system.  During  the 
first  phase  of  the  system,  the  organization  will  formulate 
the  statements  of  its  goals.  It  is  important  to  note  that 
these  goals  will  then  be  incorporated  into  all  three  of  the 
remaining  phases  of  the  selection  system.  Therefore,  the 
proposed  system  automatically  adapts  to  different  types  of 
organizations. 

Even  within  each  "type"  of  organization,  the  context  in 
which  the  project  selection  decision  takes  place  will  vary. 
Many  of  these  differences  can  be  grouped  into  differences  in 
the  structure  and  the  size  of  the  organization. 

Organizational  Structure 

In  applying  the  proposed  R&D  project  selection  system, 
the  structure  of  the  organization  will  sometimes  dictate  the 
best  way  to  implement  the  system.  For  example,  the  question 
of  what  groups  are  responsible  for  the  evaluation  of  the 
different  classes  of  projects  is  mainly  a  question  of 
organizational  structure.  The  classification  scheme  will 
often  be  dictated  by  the  organization's  structure.  On  the 
other  hand,   the  functioning  of  the  main  committee  should  be 
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reasonably  independent  of  the  organizational  structure. 

A  good  summary  of  possible  views  of  the  structure  of 
organizations  is  presented  by  White  [W-3].  White  has 
identified  four  major  types  of  organizational  structures  as 
follows: 


1)  discipline, 

2)  product/project, 

3)  stage/phase,  or 

4)  matrix. 


A  discipline  structure  groups  together  the  technical 
disciplines  in  order  to  create  a  so-called  "center  of 
excellence."  These  centers  enjoy  the  scientific  asset  of 
collective  genius  but  fail  in  respect  to  developing  market- 
able projects  since  projects  are  often  chosen  solely  on 
scientific  criteria.  Many  corporations  are  organized  in 
this  way.  For  instance,  a  company  may  be  divided  into 
electronics,  aerospace,  and  materials  groups. 

Product/project  structures  offer  the  advantage  of 
combining  necessary  disciplines  (scientists,  engineers, 
marketers,  etc.)  together  to  insure  a  smooth  transition  of 
each  project  throughout  the  various  phases  of  its  life.  An 
example  of  a  product/project  structure  could  be  a  corp- 
oration with  automotive,  computer,  and  helicopter  divisions. 

The  stage/phase  structure  is  designed  to  allow  those 
scientists  or  engineers  who  are  better  skilled  in  a  certain 
phase  of  the  R&D  process  (basic,   applied,   development)   to 
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concentrate  on  this  phase.  This  structure  is  more  likely  to 
be  found  in  a  research-oriented  firm  which  has  basic 
research,  development,  and  manufacturing  divisions.  Baker 
and  Sweeney  [B-2]  expand  on  the  differences  between  what 
they  call  phase-dominated  versus  project-dominated  organi- 
zational models.   See  Figure  7. 

Finally,  the  matrix  organizational  structure  is  any 
combination  of  the  above  three  structures.  Most  organ- 
izations would  probably  fall  under  this  category  since  there 
are  an  endless  number  of  structures  that  fall  into  this 
"matrix"  category. 

The  implications  for  the  application  of  the  proposed 
project  selection  system  are  clear.  If  a  discipline 
structure  is  used,  classifying  projects  by  discipline  might 
be  the  obvious  and  best  classification  scheme  to  use.  This 
would  allow  each  of  the  discipline  groups  to  evaluate  the 
R&D  projects  in  their  specific  areas.  Not  only  is  each 
group  uniquely  qualified  to  evaluate  their  own  projects,  but 
they  are  the  ones  with  all  the  evaluative  information  to 
make  such  comparisons.  Additionally,  if  the  organization  is 
structured  by  discipline,  then  this  may  be  an  indication 
that  a  balance  of  R&D  projects  by  discipline  may  be  a 
reasonable  organizational  goal.  Similar  arguments  can  be 
made  for  considering  product/project  or  stage/phase  classi- 
fication schemes  based  on  the  structure  of  the  organization. 

Another  difference  in  the  organizational  structure   is 

9  0 


Phase-Dominated  Structure 


DEVELOPMENT: 


Definition  of  N**tU 


ENGINEERING: 


Definition  of  Needs 

A 


PROJECT  RESULTS- 


PROJECT  RESULTS 


PROJECT  RESULTS 


MANUFACTURING: 
Definition  of  Needl 

4 


PROJECT  RESULTS 


Project-Dominated  Structure 

Recognition  of  organizational  opportunity 


Idea  generation 


Idea  submission 


Project  selection 


Project  activity,  resource  allocation, 
and  scheduling  across  phases 


I 

Research               >  Development- 

! 

F 

New  or  improved  product  or  process 


FIGURE  7:  Phase-Dominated  vs.  Project-Dominated  Models. [B-5] 
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the  amount  of  centralization  of  the  R&D  function.  Most 
notably,  an  IRI  study  conducted  for  the  National  Science 
Foundation  [N-0.3]  concluded  that  the  more  centralized  the 
R&D  function,  the  greater  the  tendency  to  perform  basic 
research.  If  this  tendency  is  part  of  the  goal  structure  of 
the  organization,  this  presents  no  problem,  but  organ- 
izations with  highly  centralized  or  decentralized  R&D 
functions  should  be  aware  of  this  trend.  This  is  especially 
true  since  IRI ' s  1987  survey  [M-7.5]  indicates  a  recent 
trend  toward  decentralized  R&D  efforts  that  could  have  a 
deleterious  effect  on  basic  research. 

Therefore,  the  structure  of  the  organization  can  often 
dictate  the  best  classification  scheme  to  use  and  how  the 
project  selection  system  is  utilized.  In  the  many  matrix 
organizations,  the  decision  may  not  be  immediately  clear  but 
decision  makers  should  consider  the  structure  of  the  organi- 
zation when  determining  an  appropriate  classification 
scheme . 

Size  of  Organization 

A  few  changes  could  be  made  to  the  proposed  project 
selection  system  depending  on  the  size  of  the  organization. 
Obviously,  the  size  of  the  committee  will  vary  with  the 
number  of  key  players  in  the  R&D  process--a  function  of  the 
size  of  the  R&D  effort.    For  a  very  small  organization,  the 
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committee  could  consist  of  as  few  as  two  people  and  the  use 
of  a  group  decision  making  technique  would  not  be  necessary. 
On  the  other  hand,  if  the  organization  is  very  large  and 
geographically  dispersed,  the  Delphi  Method  may  be  the  only 
practical  decision  making  mode. 

Additionally,  it  may  not  be  justified  for  a  small 
organization  to  carry  out  the  phases  of  the  proposed 
selection  system  concerning  the  classification  of  available 
projects  and  selecting  projects  within  each  group.  If  the 
resources  or  expertise  of  the  organization  is  very  limited, 
so  might  be  the  number  of  alternative  projects.  An  organ- 
ization that  performs  R&D  by  contract  only,  and  thus  must 
restrict  its  alternatives  to  available  contracts,  might  also 
face  a  limited  number  of  alternative  projects.  For  these 
situations,  an  application  of  a  portfolio  selection  model 
with  all  alternative  projects  included  in  the  model  may  be 
more  appropriate. 

Therefore,  the  proposed  R&D  project  selection  model  is 
designed  to  guide  organizations  in  developing  a  sensible 
project  selection  system.  No  one  best  system  can  ever  be 
designed  for  all  organizations  or  even  all  selection 
decisions  within  the  same  organization.  Many  variables  are 
involved  in  each  selection  decision  that  must  be 
appropriately  modeled  into  the  system. 

Some  of  the  most  important  considerations  are  discussed 
in   this  chapter.    The  careful  selection  of  an  appropriate 
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classification  scheme  will  help  organizations  more 
effectively  use  R&D  project  selection  methods  and  retain  a 
strategic  balance  of  R&D  efforts.  Since  the  type  of 
organization  determines  the  generic  goals  of  the  decision 
makers,  these  considerations  play  a  major  role  in  the 
selection  process.  Finally,  the  implementation  of  Phases  II 
and  III  are  often  dictated  by  the  structure  and  size  of  the 
organization. 
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CHAPTER  SEVEN: 
SUMMARY  AND  CONCLUSIONS 

The  need  for  more  practical  approaches  to  research  and 
development  project  selection  is  well  noted  in  literature 
and  practice.  Most  models  developed  in  the  past  have 
focused  on  the  quantification  of  the  selection  process. 
While  these  models  do  improve  the  decision-makers'  under- 
standing of  the  projects  and  the  selection  criteria,  the 
models  themselves  cannot  solve  the  problem.  The  selection 
decision  contains  too  many  subjective  factors  that  cannot  be 
completely  "modeled."  With  few  exceptions,  the  individuals 
involved  in  the  R&D  process  are  unwilling  to  give  up  the 
responsibility  for  the  final  allocation  of  resources  to 
projects  to  a  pure  mathematical  selection  method. 

Consequently,  the  recent  literature  has  begun  to  focus 
on  needs  of  the  decision-makers  themselves.  More 
researchers  are  testing  the  use  of  various  group  decision 
making  methods  in  this  environment.  These  tests  are 
revealing  the  value  of  this  approach—especially  when 
combined  with  a  more  analytical  approach  to  stimulate 
discussion  among  the  participants. 

The  selection  system  proposed  in  this  report  offers 
several  advantages  over  purely  analytical  models.  First  of 
all,   the   establishment   of  the  goals  of   the  organization 
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before  the  selection  process  is  begun  insures  that  the 
decision-makers  concentrate  on  the  needs  of  the  organization. 
During  this  process,  the  individuals  representing  the 
various  units  within  the  organization  work  together  to  a 
common  understanding  of  the  goals  of  the  organization.  This 
group  effort  offers  much  longer-term  advantages  in  the  form 
of  improved  communication,  cooperation,  and  motivation. 

Secondly,  the  classification  of  projects  improves  the 
selection  system  in  several  ways.  Since  most  management 
science  models  are  most  applicable  when  dealing  only  with 
similar  projects,  the  classification  of  projects  before 
choosing  or  ranking  insures  the  appropriate  use  of  these 
models.  Additionally,  the  decision-makers  are  forced  to 
consider  the  need  for  a  balanced  R&D  portfolio.  Balancing 
an  R&D  portfolio  involves  insuring  that  each  of  the 
different  classes  of  projects  are  represented  according  the 
the  objectives  of  the  organization.  A  properly  balanced 
portfolio  should  maximize  the  returns  to  the  organization 
while  minimizing  the  risk  of  project  failure. 

Finally,  the  combination  of  the  use  of  a  portfolio 
model  and  a  group  decision-making  technique  offers  the 
advantages  of  both  tools.  The  portfolio  model  is  used  as  an 
analytical  tool  to  provide  useful  information  on  the  trade- 
offs involved  in  various  portfolios  that  otherwise  would  not 
be  available.  The  group  decision-making  technique  is  used 
with  the   recognition  that  the  individuals   are   the   final 
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decision-makers  and  that  a  structure  for  the  discussion  is 
necessary. 

The  final  output  of  this  selection  system  is  that  set 
of  projects  that  will  best  meet  the  goals  of  the  organ- 
ization. The  system  recognizes  the  needs  of  the  decision 
makers  for  a  tool  to  assist  in  making  complex  trade-offs 
between  goals  while  recognizing  their  ultimate  responsi- 
bility for  the  selection  decision.  This  system,  once 
adapted  to  the  specific  organization,  will  serve  as  an 
effective  tool  in  aiding  decision  makers  faced  with  the 
selection  of  research  and  development  projects. 

Future  research  in  this  area  must  be  concerned  with  the 
application  of  the  proposed  selection  system  for  various 
types  of  organizations  and  sets  of  project  alternatives. 
Several  representative  applications  could  be  identified  and 
the  organizational  goals,  classification  scheme,  project 
selection  models,  portfolio  selection  model,  and  group 
decision  making  technigue  would  be  selected  to  best  meet  the 
needs  of  that  particular  organization.  These  applications 
would  give  insight  to  the  similarities  and  differences  in 
the  manner  in  which  the  proposed  system  is  applied.  These 
similarities  and  differences  could  then  be  categorized  to 
form  a  type  of  expert  system  to  aid  R&D  managers  in 
determining  how  to  best  apply  the  proposed  system  to  their 
particular  situation. 
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ABSRACT 

Research  and  development  (R&D)  is  defined  as  the  dis- 
covery and  application  of  new  knowledge  and  the  new  appli- 
cation of  existing  knowledge.  The  importance  of  R&D  to 
individual  organizations  and  the  economy  as  a  whole  is 
evidenced  by  the  attention  in  literature  given  to  the  R&D 
project  selection  decision.  Although  many  models  have  been 
proposed  to  deal  with  the  complexities  of  this  decision 
process,  very  few  have  been  actually  employed.  This  paper 
proposes  a  more  practical  selection  system  based  on  the 
classification  of  R&D  projects.  The  four  stages  of  the 
system  are  as  follows:  1)  establish  organizational  goals, 
2)  classify  projects,  3)  choose/rank  projects  within  each 
class,  and  4)  pool  the  projects  and  select  a  portfolio.  The 
classification  of  projects  insures  a  more  appropriate  use  of 
R&D  project  selection  models  and  helps  to  establish  a  more 
balanced  R&D  portfolio.  The  combined  use  of  an  analytical 
model  and  a  group  decision  making  technique  is  used  to 
select  the  optimum  portfolio.  The  model  provides  the 
decision  makers  with  a  format  for  the  analysis  of  trade-offs 
between  projects,  goals,  and  resource  limitations.  The 
group  technique  offers  additional  organizational  benefits  in 
terms  of  improved  communication,  cooperation,  and 
motivation. 


